CN118019958A - Map data processing, map data construction method, map data processing apparatus, vehicle, and computer-readable storage medium - Google Patents

Abstract

A map data processing method includes: obtaining observation data (102) of a local scene of a traffic scene currently collected by a sensor of a vehicle; determining first description information (104) of the local scene based on the observation data; obtaining map data (106) of the traffic scene, wherein the map data includes second description information generated based on historical observation data of the global scene of the traffic scene; matching the first description information with the second description information to obtain a matching result (108); and updating the map data (110) according to the matching result and the observation data of the traffic scene currently collected. A map data construction method, a map data processing device (400), a map data construction device (500), a vehicle (600), and a computer-readable storage medium are also provided.

Description

Map data processing, map data construction method, device, vehicle and computer readable storage medium Technical Field

The embodiments of the present application relate to the field of autonomous driving technology, and more specifically, to a map data processing, map data construction method, device, vehicle, and computer-readable storage medium.

Background technique

As a new technology, autonomous driving is a hot topic in the automotive industry. More and more autonomous driving vehicles are being developed and put into use. In some scenarios, autonomous driving vehicles can collect information about the surrounding environment through sensors and navigate the vehicle through map data of the vehicle's current environment, thereby achieving autonomous driving.

Summary of the invention

In view of this, the embodiments of the present application provide a map data processing, map data construction method, device, vehicle and computer-readable storage medium to solve the problem of cumbersome user operations and impact on vehicle driving safety caused by inefficient map data management in related technologies.

In a first aspect, a map data processing method is provided, the method comprising:

Obtaining observation data of local scenery of the traffic scene currently collected by the vehicle's sensor;

Determine first description information of the local scene based on the observation data;

Acquiring map data of the traffic scene, wherein the map data includes second description information generated based on historical observation data of global scenery of the traffic scene;

Matching the first description information with the second description information to obtain a matching result;

The map data is updated according to the matching result and the observation data of the currently collected traffic scene.

In a second aspect, a method for constructing map data is provided, the method comprising:

Controlling the vehicle to move in a traffic scene, and obtaining observation data of the global scene of the traffic scene collected by the vehicle's sensor;

Determine description information of the global scene of the traffic scene based on the observation data;

The description information of the global scene is stored as map data of the traffic scene; the description information is used to determine the position information of the vehicle in the traffic scene and determine whether the map data needs to be updated when the vehicle enters the traffic scene again.

In a third aspect, a map data processing device is provided, the device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor implements the map data processing method embodiment described in the first aspect when executing the computer program.

In a fourth aspect, a map data construction device is provided, the device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor implements the map data construction method embodiment described in the second aspect when executing the computer program.

In a fifth aspect, a vehicle is provided, the vehicle comprising the map data processing device described in the third aspect, and/or the map data constructing device described in the fourth aspect.

In a sixth aspect, a computer-readable storage medium is provided, wherein a plurality of computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the steps of the map data processing method described in the first aspect are implemented.

In a seventh aspect, a computer-readable storage medium is provided, wherein a plurality of computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the steps of the map data processing method described in the second aspect are implemented.

By applying the solution provided by the present application, the vehicle can obtain the observation data of the local scenery of the traffic scene currently collected by the sensor, and determine the first description information of the local scenery based on the observation data; and the map data of the traffic scene includes the second description information generated based on the historical observation data of the global scenery of the traffic scene; therefore, based on whether the first description information matches the second description information, it can be determined whether to update the map data. Therefore, the solution of the present application can efficiently manage the map data of the traffic scene where the vehicle is located, and can timely and automatically update the existing map data using the observation data currently collected by the sensor, thereby reducing the operation of users manually updating the map data and ensuring the safe driving of the vehicle based on the updated map data.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG. 1 is a schematic diagram of a map data processing method according to an embodiment of the present application.

FIG. 2A is a schematic diagram of a map data processing method according to another embodiment of the present application.

FIG. 2B is a schematic diagram of a map data construction method according to an embodiment of the present application.

FIG. 2C is a schematic diagram of a map data processing method according to another embodiment of the present application.

FIG. 3 is a schematic diagram of a map data construction method according to another embodiment of the present application.

FIG. 4 is a schematic diagram of the structure of a map data processing device used to implement the map data processing method of this embodiment of the present application.

FIG. 5 is a schematic diagram of the structure of a map data construction device for implementing the map data construction method of this embodiment of the present application.

FIG. 6 is a block diagram of a vehicle according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

In some scenarios, autonomous vehicles can use sensors to perceive information about the surrounding environment in real time to make autonomous driving decisions. In other scenarios, in order to reduce computing power and improve decision-making efficiency, autonomous vehicles can use sensors to collect information about the surrounding environment and combine it with map data of the vehicle's current environment to achieve autonomous driving.

The map data in the embodiments of this application refers to high-precision map data suitable for autonomous driving vehicles. Different from ordinary electronic maps used for navigation in daily life, such map data contains richer and more detailed information. It is used by the vehicle-oriented autonomous driving system to enable the autonomous driving system to perform environmental perception, high-precision positioning, planning and decision-making. Therefore, map data has an important impact on the safety of vehicle autonomous driving.

Based on this, the present application provides an embodiment of a map data processing method, which can efficiently manage the map data of the traffic scene where the vehicle is located, and can timely and automatically update the existing map data using the observation data currently collected by the sensor, thereby reducing the user's manual update of the map data operation and ensuring the safe driving of the vehicle based on the updated map data. The following embodiment is described in detail.

As shown in FIG1 , it is a flowchart of a map data processing method in an embodiment of the present application. The method in this embodiment can be executed by a map data processing device of a vehicle, which is implemented by software and/or hardware and can be configured in an electronic device with certain data computing capabilities. Optionally, the electronic device can be specifically configured in the vehicle or can be independent of the vehicle. The map data processing method embodiment may include the following steps:

In step 102, observation data of local objects of a traffic scene currently collected by a sensor of the vehicle is obtained.

In step 104, first description information of the local scene is determined based on the observation data.

In step 106 , map data of the traffic scene is obtained, wherein the map data includes second description information generated based on historical observation data of global scenes of the traffic scene.

In step 108, the first description information and the second description information are matched to obtain a matching result.

In step 110, the map data is updated according to the matching result and the observation data of the currently collected traffic scene.

This embodiment is applicable to scenarios where a vehicle needs to use map data to travel in a traffic scene. Using the above embodiment, the map data can be updated when the vehicle is in a traffic scene. Among them, this embodiment is applicable to a variety of traffic scenes, that is, the traffic scene can have a variety of embodiments, for example, it can be a parking lot, such as an underground parking lot or an outdoor parking lot, etc.; it can also be other traffic scenes that require map data to control vehicle driving, such as industrial parks, freight stations or docks. In such traffic scenes, the vehicle can obtain the map data of the traffic scene, and the map data can be used by the autonomous driving software, so that the autonomous driving software can make driving decisions based on the comparison of the vehicle's real-time observation data and map data.

The scene in this embodiment refers to any target in the traffic scene that can be sensed by the vehicle's sensor, and is a target that the vehicle's sensor needs to collect observation data when controlling the vehicle's driving. These scenes are in the traffic scene and can have position information in the traffic scene. Therefore, when the vehicle's sensor senses its information, these scenes can be used to indicate its position information in the traffic scene to the vehicle, so that the vehicle can determine the position information of the scene in the traffic scene based on the observation data, thereby determining the vehicle's own position in the traffic scene, and then being able to execute automatic driving decisions. There are many embodiments of the scene in actual applications, and one or more scenes can be identified as needed from the observation data collected by the vehicle's sensor. As an example, the scene can include lanes, parking spaces, vehicles, walls, columns, signs, lighting, pedestrians or parking lot gates, etc., or it can also be monitoring equipment arranged in the traffic scene, markers on walls, markers on columns, markers on parking spaces, etc., or it can also include light distribution in the traffic scene, etc.

The map data of the traffic scene of this embodiment can be constructed in a variety of ways. In some scenarios, the user may drive a vehicle in the traffic scene, and the vehicle's sensors collect observation data of various scenes in the traffic scene during the driving process; in other scenarios, the vehicle may automatically drive at a low speed in the traffic scene, and the vehicle's sensors collect observation data of various scenes in the traffic scene during the driving process. The map data can be constructed based on the collected observation data.

In practical applications, the vehicle may have multiple sensors, such as image sensors, lidar sensors, millimeter-wave radar sensors, or ultrasonic sensors. Based on different sensors, the vehicle's sensors may collect multiple types of observation data, such as image sensors collect image data, lidar sensors can collect point cloud data, millimeter-wave radar sensors can collect millimeter-wave data, and ultrasonic sensors can collect ultrasonic data.

In some examples, the traffic scene has a certain range, and the vehicle can travel in the traffic scene, so that the vehicle's sensors can cover the entire traffic scene and collect observation data of the entire traffic scene. This embodiment refers to the observation data of the global scene of the traffic scene. The global scene here may include all targets in the entire traffic scene that need to collect observation data. Based on the observation data of the global scene of the traffic scene, map data that can cover the entire traffic scene can be constructed.

In some examples, the map data of the traffic scene may also record the location information of all the scenes of the traffic scene in the traffic scene. The location information of this embodiment may include the geographical location information of the scene. For example, the geographical location information of the vehicle may be determined based on the information collected by the positioning sensor of the vehicle, and when the vehicle is in the geographical location information, other sensors may sense the scene, thereby determining the geographical location information of the scene. In some traffic scenes, such as underground parking lots where the satellite signal or communication signal is poor, the vehicle may not be able to accurately obtain the geographical location information. Optionally, the location information of the scene in the traffic scene may also include the relative position information of the scene in the traffic scene. For example, the position of a known geographical location information may be determined in the traffic scene as a base point, and the location information of the scene in the traffic scene may also include the relative position information of the scene relative to the base point.

In other examples, the map data of the traffic scene may include feature information of each scene in the traffic scene determined based on the observation data. For example, the feature information of the scene may include visual feature information of the scene, etc. In practical applications, based on various types of observation data collected by various types of sensors, different processing methods may be used to process the various types of observation data to determine the feature information of the global scene of the traffic scene from the observation data.

The map data in the traffic scene constructed by the vehicle can be stored locally and can be used by the vehicle when it enters the traffic scene again later. In other examples, the vehicle can also send the map data to other vehicles that can communicate, for example, by short-range wireless communication, or by P2P (peer to peer) and other methods. In other examples, the vehicle can also send the map data of the traffic scene to the server, and optionally, the server can send the map data to other vehicles that can communicate with the server.

In some examples, when a vehicle uses map data of a traffic scene for automatic driving, the vehicle's sensors may collect observation data of local scenes of the traffic scene in real time, and based on a comparison of the currently collected observation data with the observation data recorded in the map data, the vehicle's current position in the traffic scene may be determined; for example, the map data records the position information of each scene in the traffic scene, and by querying the currently collected scene and the scene that matches the map data, the vehicle's current position in the traffic scene may be determined based on the position information of the matching scene in the map data.

In actual applications, the map data of traffic scenes may face partial or complete failure; for example, the map data of traffic scenes is pre-built, and after the construction, some scenes in the traffic scenes may change. When the vehicle is driving in the changed traffic scene, the map data used does not record the newly changed scenes, which may cause the vehicle to be unable to locate its position in the traffic scene, and may make wrong decisions, causing the vehicle to be in a risky state. Or, the pre-collected observation data for certain locations of the traffic scene is relatively sparse, so the map data has less information at this location, resulting in poor positioning accuracy of the vehicle at this location when the map data is used later. Or, the environmental state when the observation data is collected is greatly different from the environmental state when the map data is used later to control the movement of the vehicle. For example, the map data constructed by collecting observation data under good light conditions during the day may cause the vehicle to be unable to efficiently and accurately locate its position in the traffic scene if it is used in a night environment. Or, the vehicle locates itself at a certain position in the traffic scene at a certain moment, and the position located by the vehicle at the next moment is quite different from the position located at the previous moment. In theory, the position difference of the vehicle at two similar moments should not be too large under normal conditions. The large difference is likely to be caused by the failure of the map data used. Based on this, these situations all indicate that the map data used at the current moment is unreliable at the current moment. In order to ensure the safety of vehicle driving, it is necessary to promptly find out whether the map data needs to be updated.

In this embodiment, the vehicle can obtain the observation data of the local scenery of the traffic scene currently collected by the sensor, and determine the first description information of the local scenery based on the observation data; and the map data of the traffic scene includes the second description information generated based on the historical observation data of the global scenery of the traffic scene; therefore, based on whether the first description information matches the second description information, it can be determined whether to update the map data. Through the solution of this embodiment, the vehicle can automatically and timely update the map data without the need for user intervention to perform operations. Since the map data can be updated in a timely manner, the driving safety of the vehicle can also be guaranteed.

In this embodiment, whether the map data needs to be updated is determined by using the first description information of the currently collected local scene and the second description information in the map data. The first description information of the currently collected local scene can represent the current state of the data currently collected for the scene, such as the time state, the state of one or more scenes or the weather state, etc., and the second description information in the map data can represent various states of the traffic scene when the map data is constructed, such as the time state, the state of one or more scenes or the weather state, etc. Therefore, based on the comparison between the first description information and the second description information, it can be determined whether the map data needs to be updated.

In some examples, a vehicle moves in a traffic scene and collects observation data of local scenery in real time. The first description information of the local scenery can be determined based on the observation data, that is, the first description information is determined by the vehicle while it is currently collecting data; and the acquired map data of the traffic scene includes second description information generated based on historical observation data of the global scenery of the traffic scene. Relative to the first description information, the second description information was generated before the current moment, and is historical description information generated by the observation data of the scenery in the traffic scene collected by the vehicle and/or other vehicles before. It can be understood that the generation method of the first description information and the second description information can be the same. Next, various implementation methods of the description information are illustrated.

In some examples, the description information may include features extracted from the observation data; for example, the vehicle sensor may include an image sensor, etc., the visual device may provide the appearance information of the scene, and the binocular vision may also provide the depth information of the scene within a certain range, for example, it may provide dense point cloud information within a certain range around the vehicle. The vehicle sensor may also include a laser radar, etc., which has high measurement accuracy and can provide point cloud information within a 360-degree range. Based on this, the features of the scene in the traffic scene may be extracted from the image data or the point cloud data. The feature extraction method is not limited in this embodiment, and may be implemented by a neural network or the like. For example, ORB features may be extracted and matched; wherein the ORB features may be extracted in real time, and the extracted features may be subjected to optical flow tracking and descriptor matching. Among them, the point cloud data collected by the laser radar may also be combined to determine the depth feature information of the pixel points in the image, and the features of the point cloud may also be extracted from the point cloud data, etc. These features may be used as description information and stored as part of the map data, and when the map data is subsequently used for navigation, feature matching may be performed with the observation data collected by the vehicle in real time.

In some examples, the description information may also include key observation data among all the collected observation data, which are referred to as key observation data in this embodiment, wherein one or more key observation data may be configured as needed. Optionally, the key observation data may be data that has an impact on the movement of the vehicle, such as scenes such as lanes and parking spaces in the traffic scene, which may have a greater impact on the driving or positioning of the vehicle, and the observation data of these scenes may be used as key observation data. Alternatively, certain scenes in the traffic scene may provide a greater role in the decision-making of the vehicle movement, and the observation data of these scenes may also be used as key observation data. For example, the visual information of a certain scene in the traffic scene is relatively rich, and these rich visual information may provide a positive effect on the control of the vehicle, such as being easier to identify and thus providing positioning information more accurately, so these scenes may be used as key observation data. Alternatively, the key observation data may be determined in combination with the vehicle's motion posture. For example, if the vehicle is in a state where the driving trajectory changes, such as turning, it indicates that the current scene has changed significantly and needs to be given certain attention. Therefore, the observation data collected when the vehicle's motion posture undergoes a preset change may be used as key observation data.

In some examples, the key observation data includes: the observation data collected by the sensor within the target sampling period when the vehicle posture undergoes a preset change. Optionally, the preset change in the vehicle posture of this embodiment can be implemented in a variety of ways as needed, for example, it can include a preset change in the vehicle's driving direction, where the preset change can include a change in the driving direction angle greater than a set angle threshold, such as the vehicle turning left or right after driving forward, or the vehicle driving forward and then reversing. Optionally, it can also include a preset change in the vehicle's driving trajectory, such as the driving trajectory changing from a straight line to a curve.

In some examples, the vehicle's sensor collects the observation data according to a preset sampling period; the descriptive information includes key observation data, and the key observation data characterizes the observation data collected by the sensor within a target sampling period. The vehicle's sensor may include one or more of an image sensor, a lidar sensor, a millimeter-wave radar sensor, or an ultrasonic sensor, and the corresponding key observation data includes one or more of the following data collected within the target sampling period: image data collected by the image sensor within the target sampling period; point cloud data collected by the lidar sensor within the target sampling period; millimeter-wave data collected by the millimeter-wave radar sensor within the target sampling period; ultrasonic data collected by the ultrasonic sensor within the target sampling period.

In some examples, key observation data can be determined based on changes in traffic scenes during vehicle travel. For example, the target sampling period is determined based on the observation data of multiple sampling periods. The vehicle's sensors collect observation data according to the sampling period, and the observation data of the target sampling period can be determined as key observation data based on the observation data of multiple sampling periods as needed. As an example, whether there is key observation data can be determined by comparing the observation data of multiple sampling periods. Based on this, whether there is data that can be used as key observation data can be found out through changes in the observation data of multiple sampling periods.

As an example, the key observation data may be the observation data of the target sampling period when the data changes greatly among the observation data of the multiple sampling periods. For example, if a large change is found in the scenery from one position to another in the traffic scene, the manifestation of this large scene change in the observation data collected by the vehicle sensor is a large change in the characteristics of the scenery in the observation data; for example, there is a pillar between two adjacent parking spaces, and the data collected by the vehicle's sensor for the two adjacent parking spaces can be found to have a large change in the characteristics of the data from the observation data of one parking space to the observation data of the pillar and then to the observation data of another parking space. These changes can be paid attention to in order to control the movement of the vehicle more stably and safely. Based on this, the key observation data for the target sampling period can be determined based on the characteristic changes of the scenery in the observation data of the multiple sampling periods. In other examples, the data change of this embodiment may also refer to the change between the observation data of the local scene of the traffic scene currently collected and the historical observation data of the local scene in the map data; for example, for a certain scene in the traffic scene, the data features of the observation data of the scene currently collected are significantly different from the data features of the historical observation data of the scene in the map data, indicating that the current state of the scene is significantly different from the historical state. Therefore, the observation data of the scene currently collected can be used as key observation data.

In other examples, the key observation data may also be the observation data of a target sampling period when the number of data features is greater than a set threshold among the observation data of multiple sampling periods, wherein the set threshold can be configured as needed; based on the observation data of multiple sampling periods, it is determined that the number of features of the scene is large, and since the features of the scene are relatively rich, rich information can be provided in subsequent processing, and therefore it can be used as the key observation data.

In other examples, the key observation data may also be the observation data of a target sampling period when the number of data features is less than a set threshold among the observation data of multiple sampling periods, wherein the set threshold can be configured as needed; it is determined based on the observation data of multiple sampling periods that the number of features of the scene is small. Due to the small number of features, the observation data of the scene at this location may not be able to be accurately located and identified by the autonomous driving system. Therefore, the observation data of these scenes with small numbers can be regarded as key observation data and given attention.

In some examples, the data features of the observation data collected during the target sampling period meet the preset data feature conditions. In this embodiment, key observation data can be determined based on data features, and observation data that meet the preset data feature conditions are found as key observation data from the observation data of multiple sampling periods according to data features, that is, the data features of the key observation data meet the preset data feature conditions. In practical applications, the preset data feature conditions can be implemented in a variety of ways as needed. As an example, it can be image data collected by an image sensor during the target sampling period, which meets the preset image texture conditions, that is, the image data as key observation data, whose image texture meets the preset image texture conditions; the preset image texture conditions can be implemented in a variety of ways, for example, the image texture conditions can include the number of image texture features matching the set number threshold, the image texture feature density matching the set density threshold, or the image texture feature arrangement matching the set arrangement rule, etc. In other examples, the point cloud data collected by the lidar sensor within the target sampling period may satisfy the preset point cloud distribution conditions, that is, the point cloud data serving as key observation data may have a point cloud distribution that satisfies the preset point cloud distribution conditions; the preset point cloud distribution conditions may be implemented in a variety of ways, for example, the point cloud distribution conditions may include the number of point cloud points in the point cloud matching a set number threshold, the point cloud point density of the point cloud matching a set density threshold, or the distribution of the point cloud matching a set distribution law, and so on.

In some examples, the description information may also include tag information, that is, the scene in the traffic scene corresponds to the tag information. In actual applications, one or more tag information may be configured for the scene in the traffic scene as needed, and map data matching may be performed quickly based on the tag information.

In some examples, the label information may include: label information for characterizing the category to which the specific scene in the traffic scene belongs. The specific scene may be a global scene in the traffic scene, that is, all scenes have label information characterizing the category to which they belong; in other examples, the specific scene may be part of the scene in the traffic scene, that is, part of the scene has label information characterizing the category to which it belongs, while part of the scene does not have the label information. In practical applications, the specific scene with label information of the category to which it belongs can be configured as needed, for example, it can be some key scenes in the traffic scene, such as scenes that help to locate the vehicle, taking the parking scene as an example, including but not limited to parking spaces, markers on the wall or lighting, etc., and can also include scenes that affect the driving safety of the vehicle, such as pedestrians, lanes, etc. In practical applications, multiple categories can be set as needed, for example, specific scenes in the traffic scene can be classified based on multiple different classification methods; for example, based on whether the scene has a fixed position attribute, the category can include fixed scenes and non-fixed scenes, such as walls, parking spaces or columns, etc., which belong to fixed positions; for example, pedestrians, vehicles or light, etc., which belong to non-fixed scenes. Alternatively, the classification method may also include the category of the location of the scene, such as the scene on the ground or in space, such as lanes or parking space markers, etc., which belong to the ground category, and lighting or monitoring equipment, etc., which belong to the scene in space. In other examples, multiple other categories may also be included. For example, different categories may be set based on different traffic scenes. For example, in a parking lot scene, the categories may include walls, columns, vehicles, parking spaces, lanes, wall markers, column markers, or pedestrians, etc. Based on the above-mentioned category settings, each scene may have one or more categories, that is, each scene may have one or more label information representing the category to which it belongs.

In some examples, the label information may also include: label information for characterizing the collection time of the observation data of the traffic scene; wherein the collection time can accurately indicate the timeliness of the map data and the environmental status when the map data is constructed. In practical applications, a variety of label information characterizing the collection time can be set as needed. As an example, label information characterizing the season to which the collection time belongs, such as label information for different seasons of spring, summer, autumn and winter, may be included. In other examples, label information characterizing the time period to which the collection time belongs may be included. The time period may be set as needed. As an example, the time period may include morning, afternoon or evening, etc.

As can be seen from the above embodiments, in some examples, the description information may include key observation data, which, as key observation data in the map data, plays a key role in the control of the vehicle. Based on this, the label information of this embodiment also includes: label information for characterizing the characteristics of the key observation data, that is, this embodiment also configures labels for the characteristics of the key observation data, so that based on the label information of the characteristics of the key observation data, it can be more quickly and accurately determined whether the map data needs to be updated based on the key observation data.

Considering that the autonomous driving system requires accurate environmental perception to correctly control the vehicle motion, among which accurate environmental perception, one important information is the appearance information of the scenery in the traffic scene. Based on this, in some examples, the features of the key observation data include the visual features of the key observation data, and the visual features of the key observation data can provide the autonomous driving system with more reliable appearance information of the scenery in the traffic scene. Optionally, the visual features include at least one of the following: SiFT features (Scale-invariant feature transform, SIFT), ORB (Oriented Fast and Rotated Brief, fast feature extraction) features or MSER features (Maximally Stable Extremal Regions, maximally stable extreme regions), etc.

In this embodiment, the matching process of the first description information and the second description information can also be implemented in multiple ways. As an example, there are multiple types of description information, which can be matched with corresponding types of description information respectively. For example, the description information can include features extracted from the observation data, so the observation book currently collected by the vehicle's sensor can extract the features of the scene, and the features of the currently extracted scene can be matched with the features of the scene in the historical observation data in the map data. In other examples, the description information can include label information, and the label information determined by the currently collected observation data can be matched with the label information in the map data; wherein, there can be multiple types of label information, and the label information of corresponding types can be matched respectively during matching, for example, the label information used to characterize the collection time is determined from the currently collected observation data, and matched with the label information used to characterize the collection time in the map data; the label information used to characterize the illumination information is determined from the currently collected observation data, and matched with the label information used to characterize the illumination information in the map data, and so on. In other examples, the descriptive information includes key observation data, and the key observation data determined by the currently collected observation data and the key observation data in the map data can be matched. For example, the key observation data determined by the currently collected observation data and the key observation data in the map data can be directly matched, or the matching can be based on the features of the key observation data or label information of the features.

Based on the matching of the first description information and the second description information, the matching result of the two can be obtained, and whether to update the map data can be determined based on the matching result. Optionally, an information matching condition can be set as needed, which is referred to as a preset information matching condition in this embodiment. The preset information matching condition is used to indicate the matching degree between the first description information and the second description information, that is, the condition indicating whether the map data needs to be updated; in actual applications, the preset information matching condition can be flexibly configured as needed. For example, there are multiple first description information and second description information. Therefore, the preset information matching condition can include any combination of matching conditions of one or more description information.

As an example, the preset information matching condition may be a condition related to the tag information. For example, taking the tag information of the collection time of the observation data used to characterize the traffic scene as an example, the preset information matching condition may include: the tag information of the collection time of the currently collected observation data matches the tag information of the collection time of the map data. Alternatively, the description information may include features extracted from the observation data, and the preset information matching condition may be configured based on the features. Alternatively, the description information includes tag information used to characterize the category to which a specific scene in the traffic scene belongs, and the preset information matching condition may include tag information of the category to which the specific scene determined by the currently collected observation data belongs, matching the tag information of the category of the scene in the map data, etc. Alternatively, the description information includes key observation data, and the preset information matching condition may be configured based on the key observation data, for example, the preset information matching condition may include key observation data in the currently collected observation data, matching the key observation data in the map data, etc. In other examples, the preset information matching condition may also include a condition formed by one or more combinations of tag information, features and/or key observation data, which may be set as needed in actual applications, and this embodiment does not limit this.

In some examples, if the matching result of the first description information and the second description information meets the preset information matching condition, it can be determined that the map data does not need to be updated, and the real-time position information of the vehicle in the traffic scene can be determined based on the matching result of the first description information and the second description information, and the movement of the vehicle in the traffic scene can be controlled based on the real-time position information. If the first description information and the second description information do not match, it can be determined that the map data needs to be updated, and the map data can be updated based on the matching result and the observation data of the currently collected traffic scene.

In actual applications, there are many ways to update the map. In some examples, the updating method of the preset map data includes any of the following: replacing the second description information with the first description information; or merging the first description information with the second description information.

As an example, in actual applications, there may be a situation where the current traffic scene is significantly different from the traffic scene when the map was constructed. For example, some scenery in the current traffic scene has changed significantly, or the state of the traffic scene when the map was constructed is significantly different from the state of the current traffic scene, which results in the map data being unable to be applied to the current traffic scene. The map data is invalid. If the vehicle uses the map data for navigation, the vehicle cannot be accurately located. Based on this, the first description information can replace the second description information. Since the second description information is replaced, the map data is invalid, and new map data needs to be constructed using the currently collected observation data.

Optionally, the replacement of the second description information with the first description information represents the invalidity of map data, and based on this, it can be performed under the following conditions: the matching result does not meet the preset information matching condition, and the real-time position information of the vehicle in the traffic scene is not determined according to the matching result.

In practical applications, according to whether the matching result can determine the real-time position information of the vehicle in the traffic scene, there can be multiple implementation methods as needed. For example, the description information may include features extracted from the observation data, and the preset information matching condition can be configured based on the features. As an example, if the map data is reliable, the features of the observation data collected by the current vehicle should match the features of the historical observation data in the map data. If the map data is unreliable, the features of the observation data collected by the current vehicle and the features of the historical observation data in the map data are difficult to match well, which leads to the inability of the vehicle to use the map data to achieve stable positioning in the current traffic scene, and the inability to stably and accurately obtain the current position information of the vehicle in the traffic scene. Therefore, whether the real-time position information of the vehicle in the traffic scene can be determined can be determined according to the matching of the features of the observation data collected by the current vehicle and the features of the historical observation data in the map data. For example, if the positioning cannot be stable, the vehicle continues to travel M (M can be set as needed), the number of feature matches is less than the set value, or the average number of feature matches is less than the set value, etc. Or, it can also be continuous driving L (L can be set as needed) without matching features, etc.

In some examples, when map data fails, it is necessary to reconstruct the map data of the current traffic scene. Based on this, after the matching result does not meet the preset information matching conditions and the real-time position information of the vehicle in the traffic scene is not determined according to the matching result, the method of this embodiment may also include: controlling the vehicle to move in the traffic scene to continuously collect observation data of local scenery of the traffic scene, and determining and storing descriptive information using the continuously collected observation data. Therefore, the scheme of this embodiment can control the vehicle to execute the map data construction process after determining that the map data is invalid, thereby generating available map data for the current traffic scene.

Regarding the map update method, another optional embodiment is to merge the first description information with the second description information; in actual applications, there may be a situation where some data in the map data is invalid, that is, the map data is available as a whole, but some of it needs to be updated; for example, some scenes in the traffic scene have changed, while other scenes in the map data have not changed, that is, the data of other scenes are still available. From the perspective of vehicle positioning, that is, when the vehicle movement is controlled based on the map data, when the position information of the vehicle in the current traffic scene is located based on the map data, the positioning is unstable, that is, at some times the vehicle can continue to be positioned stably, and at some times the vehicle positioning is unstable, inaccurate or the position drifts. Or, in actual applications, when constructing map data, the observation data collected for the traffic scene may not be dense enough and there are some missing. Based on this, the currently collected observation data can be used as a supplement to add the first description information to the map data. Based on this, this embodiment can merge the first description information with the second description information, that is, use the first description information of the currently collected observation data to update some invalid second description information in the map data. As an example, the fusion method includes any of the following: merging the first description information into the preset map data; or changing part of the second description information in the preset map data according to the first description information. For example, merging the first description information into the preset map data may be adding the first description information of a new scene, or adding new description information to the description information of an existing scene, etc.; changing part of the second description information in the preset map data according to the first description information may be changing the description information of part of the scene, which may include all or part of the description information of the changed scene, etc.

In some examples, the fusion of the first description information and the second description information is performed under the following conditions: the matching result does not meet the preset information matching condition, and it is determined based on the matching result that the real-time position information of the vehicle in the traffic scene does not meet the preset stable state, and the first description information is fused with the second description information. In practical applications, the preset stable state can be implemented in a variety of ways as needed, for example, the number of matching features in the matching result is greater than a set threshold, that is, the features of the currently collected local scene can match the features of the scene in the map data, and the number of matches is rich, for example, there are N features that are continuously matched, and N can be set as needed, such as 50, etc.; due to the large number of feature matches, the vehicle can use the location information of these matching features in the map data in the traffic scene to determine the current location information of the vehicle in the traffic scene. Optionally, if the map data is reliable, the vehicle can continuously determine accurate location information, and the vehicle continues to move, and the location information of the vehicle will not change significantly within two adjacent time periods; if the real-time location information of the vehicle in the traffic scene determined based on the map data changes significantly within two adjacent time periods, it can be determined that the real-time location information of the vehicle in the traffic scene does not meet the preset stable state; based on this, the preset stable state may include that within multiple set time periods, the difference in the real-time location information of the vehicle in the traffic scene is less than a set threshold; if within multiple set time periods, the difference in the real-time location information of the vehicle in the traffic scene is greater than the set threshold, that is, the real-time location of the vehicle changes significantly, and therefore does not meet the preset stable state.

As an example, the real-time position information of the vehicle in the traffic scene does not satisfy the preset stable state, including: the real-time position of the vehicle displayed on the user interface drifts. In this embodiment, the real-time position of the vehicle can be displayed on the user interface, and if the real-time position information of the vehicle in the traffic scene does not satisfy the preset stable state, the real-time position of the vehicle displayed on the user interface drifts.

As an example, the description information may include features extracted from the observation data, and whether the map data needs to be updated can be determined based on whether the features match. For example, if the map data is reliable, the features of the observation data collected by the current vehicle should match the features of the historical observation data in the map data. If the map data is unreliable, it is difficult to match the features of the observation data collected by the current vehicle with the features of the historical observation data in the map data, which will result in the vehicle being unable to use the map data to achieve stable positioning in the current traffic scene, and unable to stably and accurately obtain the vehicle's current location information in the traffic scene. Therefore, the implementation method for feature matching and whether the map needs to be updated can be set based on this.

In some examples, a traffic scene may have multiple maps, such as different maps constructed by observation data collected at different times. The map data constructed at different times may have different characteristics. For example, the light brightness in the morning and evening is different, and the light characteristics in the map data are different. Different light brightness will cause different scenery characteristics; or, the map data constructed under different weather conditions also have different characteristics. Based on this, in the case where the traffic scene has multiple maps, each map data may have usage attribute information, and the usage attribute information is determined based on the tag information; the acquisition of the map data of the traffic scene includes: displaying the usage attribute information of the multiple maps on the user interface, acquiring the usage attribute information selected by the user through the user interface; determining the map data from the multiple maps according to the usage attribute information selected by the user. The user interface in this embodiment may be a user interface of a display component of a vehicle, or a user interface of other devices that can communicate with the vehicle. The other devices are independent of the vehicle and can communicate with the vehicle. For example, they may include portable devices, such as smart phones, personal computers or PDAs, etc., and may also include wearable devices, such as smart watches, smart bracelets, smart glasses or smart helmets, etc., which are not limited in this embodiment. Based on this, before or after the vehicle enters the traffic scene, the user interface can provide the user with a selection function for multiple map data of the traffic scene, wherein the usage attribute information can be implemented in multiple ways, for example, the usage attribute information can include time, scene or weather information of the map data. Based on this, after providing a user interface to display the usage attribute information of each map data, the user can select the appropriate map data based on the usage attribute information.

In some examples, the traffic scene has multiple map data, each map data has usage attribute information, and the usage attribute information is determined based on the tag information; the acquisition of the preset map data of the traffic scene includes: matching the tag information in the first description information with the usage attribute information of the multiple map data, and determining the map data from the multiple map data according to the matching result. In this embodiment, the vehicle can automatically select from the multiple map data of the traffic scene, and based on the usage attribute information, select the map data that best matches the state of the current traffic scene of the vehicle from the multiple map data. For example, the usage attribute information may include time, scene or weather information of each map data. When selecting, the current time information, current weather information, scene information of the current vehicle location, etc. can be obtained, and then matched with the usage attribute information of each map data. Through this embodiment, when there are multiple map data in the current traffic scene, the most suitable map data can be automatically obtained, which reduces user operations, and the effect of ensuring safe driving of the vehicle can be achieved because the map data with better reliability can be obtained.

In some examples, the method may further include: sending the updated map data to the server and/or other vehicles. In this embodiment, after the vehicle updates the map data, the updated map data may be sent to the server, which stores the updated map data. The server may also send the updated map data to other vehicles that can communicate with the server as needed; or the vehicle may send the updated map data to other vehicles that can communicate with the server for use by the other vehicles.

FIG2A is a schematic diagram of another map data processing method according to an embodiment of the present application, and FIG2A includes the following steps:

In step 210, a map is constructed; the map construction process is used to construct map data of a traffic scene; in some examples, a user may drive a vehicle in a traffic scene, and during the driving process, the vehicle's sensors collect observation data of various scenes in the traffic scene; in other scenarios, a vehicle may automatically drive at a low speed in a traffic scene, and during the driving process, the vehicle's sensors collect observation data of various scenes in the traffic scene. Map data can be constructed using the collected observation data.

As shown in FIG. 2B , a schematic diagram of a map construction process of this embodiment is shown; in step 211 , the map can be constructed using the observation data collected by the vehicle's sensors;

For example, vehicle sensors may include image sensors, etc., visual devices may provide appearance information of the scene, and binocular vision may also provide depth information of the scene within a certain range, for example, dense point cloud information within a certain range around the vehicle may be provided. Vehicle sensors may also include laser radars, etc., which have high measurement accuracy and can provide point cloud information within a 360-degree range. Taking image data as an example, in FIG2B, in step 212, image data collected by the image sensor may be obtained, and in step 213, feature extraction and matching may be performed on the image data. For example, ORB features may be extracted and matched; wherein the ORB features may be extracted in real time, and optical flow tracking and descriptor matching may be performed on the extracted features. Among them, the depth information of the pixel points in the image may also be determined in combination with the point cloud data collected by the laser radar. In step 214, a three-dimensional feature map may be reconstructed, for example, linear triangulation may be used based on the extracted ORB features, and the reprojection error method may be used to remove mismatched or low-quality three-dimensional features. Based on this, the features of each object in the scene can be extracted from the observation data. These features can be used as descriptive information and stored as map data. When the map data is subsequently used for navigation, feature matching can be performed with the observation data collected in real time by the vehicle.

In step 215, key frame extraction can be performed, that is, key observation data can be obtained, for example, a series of representative images can be extracted as key frames for more complete expression of map environment information. Key frame extraction can be performed in a variety of ways, for example, during the continuous movement and collection process of the vehicle, within the target sampling period, the number of feature point tracking and matching drops to a set threshold, such as 30%, etc.; or, within the target sampling period, the number of image feature points can be less than 300; or, within the target sampling period, the posture of the vehicle changes, such as the driving distance exceeds 2 meters or the angle of the vehicle exceeds 20 degrees, etc.

In step 216, key frame labeling may be performed. For example, for all key observation data, a variety of visual features may be extracted. The visual features may include SiFT (complex geometric texture structure), ORB (rapid extraction), MSER (local stable region feature), etc. These features may also be used to configure label information, and a dictionary may also be constructed. As an example, the label of the key observation data may include a bag-of-words feature label, an overall brightness feature label, or a feature space distribution label, etc.

In step 217, user scenario labeling processing is performed, that is, more other descriptive information is configured using the observed data. For example, the descriptive information may include a time tag. For example, the time information when the map is constructed can be obtained, and a time tag can be added to the map data, such as season, month, date, time period (morning, noon, afternoon, evening, night, etc.). Among them, the usage attribute information of the map data can be configured based on the time tag, and the usage attribute information represents the time information of the map data. In other examples, the descriptive information may also include a scene tag. For example, the environmental state of the traffic scene can be determined based on the observed data, such as an indoor parking lot, an outdoor parking lot, a semi-enclosed parking lot, an office parking lot, a residential parking lot, etc. Among them, the usage attribute information of the map data can also be configured based on the scene tag, and the usage attribute information represents the environmental state of the traffic scene to which the map data belongs.

In step 218, map storage can be performed; in the map storage stage, if the original observation data is used for storage, a large amount of space resources will be consumed, and there will be a lot of redundant information. However, in the above-mentioned process, the present embodiment generates description information based on the observation data, wherein the description information includes a large number of features of the scene, including key observation data determined based on the observation data, and a large amount of label information generated based on the observation data, etc. Therefore, the present embodiment can store the description information as map data and perform long-term maintenance on the map data.

In this embodiment, the same traffic scene can store map data with different label combinations such as different time periods, different seasons, and different lighting conditions to supplement the map failure caused by changes in different scenes. In other words, one traffic scene can correspond to multiple sets of map data.

After acquiring the map data, the vehicle can use the map to perform map positioning in step 230; in step 240, the positioning state can be determined; and in step 250, the map can be updated. This is described in conjunction with a map data processing flow chart of this embodiment shown in FIG. 2C.

It can be seen from the aforementioned embodiments that one or more copies of map data can be constructed for a traffic scene. When the vehicle enters the traffic scene again, it can navigate based on the constructed map data, and it can also determine whether the map data needs to be updated. As an example, before the vehicle enters the traffic scene or when it enters the traffic scene, the map data of the traffic scene can be obtained. For example, the geographical location of the vehicle can be determined based on the positioning sensor of the vehicle, and matching map data can be queried from the local or server side based on the geographical location. Optionally, in the case where the traffic scene has multiple copies of map data, it can be automatically selected by the vehicle or selected by the user based on the usage attribute information of the map data. In the case of automatic selection, optionally, the observation data of the current traffic scene can be collected by the vehicle's sensor, and based on the description information of the currently collected observation data, it can be matched with the usage attribute information of multiple copies of map data to find out the map data that matches the current traffic scene.

Alternatively, as shown in FIG2C , in step 261, the user may specify the map data; for example, the usage attribute information of multiple maps may be displayed in the user interface, and the user may select one of the maps. Optionally, a setting function for the current position and direction of the vehicle may be provided in the user interface, and the user may set the current position and direction of the vehicle. Since the system stores maps of multiple scenes, the user may specify the map data, which can effectively improve the accuracy of the map data and save the time for automatic matching of the map data.

The vehicle may move in the traffic scene based on the map data, and the vehicle may be positioned on the map in step 262. For example, the observation data of the local scene of the traffic scene currently collected by the sensor of the vehicle may be obtained; the first description information of the local scene may be determined based on the observation data, and the first description information may be matched with the second description information in the map data to obtain a matching result. If the matching result satisfies the preset information matching condition, the real-time location information of the vehicle in the traffic scene may be determined based on the matching result, and the vehicle may be controlled to move in the traffic scene based on the real-time location information.

Among them, the map data may be completely or partially invalid and need to be updated. In step 263, positioning quality monitoring can be performed to determine the quality of the vehicle's positioning based on the map data, based on which, in step 208, the map status can be judged. As an example, after the vehicle selects the map data, the vehicle will start driving based on the map data, and based on the observation data of the local scenery of the traffic scene currently collected by the vehicle's sensor; based on the observation data, the first description information of the local scenery is determined; for example, the real-time collected images and point cloud data can be feature extracted, and the extracted features can be used as the first description information, which can be matched with the features in the map data and the posture can be solved. When the positioning program is successfully positioned, normal automatic driving control can be performed, and the observation data of the local scenery of the traffic scene currently collected by the vehicle can also provide corresponding environmental prior information.

In step 263, positioning quality monitoring can be performed, that is, after the vehicle is positioned using the map, the map status can be monitored. As an example, the map positioning status can be set to multiple states as needed, as an example, it can include normal positioning, that is, the map data is valid and reliable, and no update is required. The positioning status can also include unstable positioning, that is, the description information of some scenes in the map data may be invalid, resulting in the vehicle sometimes being reliably positioned and sometimes unreliably positioned when using the map data for positioning. In this case, the map data needs to be updated. The positioning status can also include positioning failure, that is, the map data cannot be used for reliable positioning of the vehicle, the map data is basically invalid, and the map data of the traffic scene needs to be reconstructed.

Optionally, multiple implementations may be used to characterize the positioning state. As an example, the judgment condition for normal positioning may include: the number of feature matches is greater than a set threshold, that is, the features of the currently collected local scene can match the features of the scene in the map data, and the number of matches is abundant, for example, there are N features that are continuously matched, and N can be set as needed, such as 50.

As for the unstable state and positioning failure, that is, the vehicle is not in normal positioning, there are multiple optional judgment methods, for example, the number of feature matches is small, such as the vehicle continuously travels M (M can be set as needed), the number of feature matches is less than the set value, or the average number of feature matches is less than the set value, etc. Alternatively, it can also be continuous travel L (L can be set as needed) without matching features. When it is not in the normal positioning state, the map state judgment of step 208 can be further performed.

In step 208, a map status judgment may be performed. As an example, a judgment may be made in combination with a variety of tag information. For example, it may be judged by time and scene tags. If it is found that the time tag difference is too large, for example, the time change is greater than the set number of days; or the time period does not match, for example, the time tag of the map data indicates that the construction time period is morning, while the tag information determined by the currently collected observation data is dusk, and the time periods of the two vary greatly; or, it may be a weather mismatch, for example, the time tag of the map data indicates that the weather state at the time of construction is sunny, while the tag information determined by the currently collected observation data is cloudy, etc.; if the time tag difference is too large, it will trigger map reconstruction, that is, the map construction of step 210 may be executed. It can be understood that this embodiment triggers map construction again, and the constructed map data is an update of the existing map data.

Optionally, if there is no abnormality in the time tag information and the scene tag information, a judgment can be made based on the key observation data and/or the tag information of the key observation data; for example, the key observation data can be determined for the currently collected observation data, and the description information, such as the tag information of the key observation data, can be determined based on the key observation data. The tag information can include visual features, and features such as SiFT (complex geometric texture structure), ORB (rapid feature extraction), and MSER (local stable region feature) can be extracted, and matched and judged with the tag information of the map data and/or the key observation data. If the key observation data currently collected differs too much from the key observation data in the map data, or the tag information of the key observation data currently collected differs too much from the tag information of the key observation data in the map data, the reconstruction of the map data can be triggered, that is, the map construction of step 210 can be executed.

In step 204 of this embodiment, map storage may be performed. Optionally, in order to prevent the problem of too much map data, the map data of the same traffic scene may be automatically merged and updated according to the time tag of the map data. For example, if the time tag of the newly constructed map data is the same as the time tag of the existing old map data, the newly constructed map data may replace the old map data. Alternatively, if the time tag of the newly constructed map data is summer, and the time tag of the old map data is autumn, both the new and old map data may be retained.

The solution of this embodiment can be applied to a variety of practical scenarios. For example, the method of this embodiment can be used to solve the automatic parking problem in the vehicle's assisted driving function, such as in the automatic parking system of different scenarios such as commonly used industrial parks, residential communities, campuses, etc., to solve the problem of map failure.

For example, the solution of this embodiment can be used as an autonomous driving solution for the last 100 meters of real-life scenarios in the vehicle field, and can be applied to the vehicle's assisted parking system. Through the above-mentioned map failure judgment, triggering users to perform map updates, map reconstruction, map switching and other map data management methods, the long-term operation stability of the autonomous driving system is improved. The assisted parking system usually requires users to perform a single training for a scene, but the feature map of a single training has a certain timeliness. With changes in time, weather, light, seasons, etc., the features in the map data will gradually become invalid. Through an effective map failure management method, the effective use time of the parking system can be improved, and the tedious operation of users frequently building maps can be reduced.

The scheme of this embodiment can realize the optimized management of the parking system map. For example, in the map construction stage, when extracting the features of the observation data, the time characteristics of the map data can be marked by automatically capturing the complete timeliness information of the map data; when the map is constructed, according to the set conditions, such as distance and angle thresholds, different key observation data can be extracted to record the characteristics of the traffic scene, thereby generating a series of unique tags. By describing the timeliness of the features of the map data and the characteristics of the key observation data, the label attributes of the current traffic scene can be accurately expressed. When coming to the same traffic location again, only a small number of features are matched, but the observation data and the key observation data are highly similar, it can be determined that the current traffic scene has changed, triggering the user to perform a map data update strategy; when only a very small number of features are matched, and the key observation data and the real-time collected observation data are significantly different, it can be determined that the current map is in an invalid state; the map data needs to be re-collected and added to the map library; through the above map update strategy, the use efficiency of the parking map and the robustness of the scene changes can be effectively improved.

Automatic parking technology is one of the technologies that the industry has invested heavily in and is expected to achieve L4 level mass production the fastest. By promoting automatic parking technology, huge product benefits will be brought. This embodiment can achieve effective management of automatic parking map failure, and can effectively improve the stability of the map positioning function during parking and the user experience of map failure.

As shown in FIG3 , the present application also provides an embodiment of a method for constructing map data, which may include the following steps:

In step 302, the vehicle is controlled to move in a traffic scene, and observation data of the global scene of the traffic scene collected by sensors of the vehicle is obtained.

In step 304, description information of the global scene of the traffic scene is determined based on the observation data.

In step 306, the description information of the global scene is stored as map data of the traffic scene; the description information is used to determine the position information of the vehicle in the traffic scene and whether the map data needs to be updated when the vehicle enters the traffic scene again.

It can be seen from the above embodiments that the map data constructed in this embodiment includes description information of the global scenery of the traffic scene. Therefore, when the vehicle re-enters the traffic scene, the description information can be used to determine the position information of the vehicle in the traffic scene and determine whether the map data needs to be updated. Therefore, the vehicle can efficiently manage the map data of the traffic scene where the vehicle is located, and can timely and automatically update the existing map data using the observation data currently collected by the sensor. Therefore, it not only reduces the operation of users manually updating map data, but also can ensure the safe driving of the vehicle based on the updated map data.

In some examples, the sensor collects the observation data according to a preset sampling period;

The description information includes key observation data, where the key observation data represents the observation data collected by the sensor within a target sampling period.

In some examples, the key observation data includes: the observation data collected by the sensor within a target sampling period when a preset change occurs in the vehicle posture.

In some examples, the sensor includes one or more of the following sensors: an image sensor, a lidar sensor, a millimeter wave radar sensor, or an ultrasonic sensor.

In some examples, the key observation data includes one or more of the following:

Image data collected by the image sensor during a target sampling period;

Point cloud data collected by the LiDAR sensor during the target sampling period;

Millimeter wave data collected by the millimeter wave radar sensor during the target sampling period;

Ultrasonic data collected by the ultrasonic sensor during the target sampling period.

In some examples, data features of the observation data collected within the target sampling period meet preset data feature conditions.

In some examples, image data collected by the image sensor within a target sampling period meets a preset image texture condition.

In some examples, point cloud data collected by the lidar sensor within a target sampling period meets preset point cloud distribution conditions.

In some examples, the target sampling period is determined based on the observed data of a plurality of the sampling periods.

In some examples, the description information includes tag information;

The tag information includes one or more of the following:

Label information used to characterize the category to which a specific scene in the traffic scene belongs;

Label information used to characterize the collection time of the observation data of the traffic scene.

In some examples, the tag information also includes:

Label information used to characterize the features of the key observation data.

In some examples, the characteristics of the key observations include visual characteristics of the key observations;

The visual feature includes at least one of the following: a SiFT feature, an ORB feature or an MSER feature.

The above method embodiments can be implemented by software, hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the image processing processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, as shown in Figure 4, it is a hardware structure diagram of the map data processing device 400 for implementing this embodiment. In addition to the processor 401 and the memory 402 shown in Figure 4, the map data processing device used to implement the map data processing method embodiment in the embodiment can also include other hardware according to the actual function of the map data processing device, which will not be described in detail.

In this embodiment, the processor 401 implements the following steps when executing the computer program:

Obtaining observation data of local scenery of the traffic scene currently collected by the vehicle's sensor;

Determine first description information of the local scene based on the observation data;

Acquiring map data of the traffic scene, wherein the map data includes second description information generated based on historical observation data of global scenery of the traffic scene;

Matching the first description information with the second description information to obtain a matching result;

The map data is updated according to the matching result and the observation data of the currently collected traffic scene.

In some examples, the processor 401 further implements the following steps when executing the computer program:

If the matching result satisfies a preset information matching condition, the real-time position information of the vehicle in the traffic scene is determined according to the matching result, and the movement of the vehicle in the traffic scene is controlled based on the real-time position information.

In some examples, the processor 401 performs the step of updating the map data according to the matching result and the observation data of the currently collected traffic scene, including:

If the matching result does not meet the preset information matching condition, the map data is updated according to the first description information.

In some examples, the map data is updated in any of the following ways:

Replace the second description information with the first description information; or,

The first description information is merged with the second description information.

In some examples, replacing the second description information with the first description information is performed under the following conditions:

The matching result does not satisfy a preset information matching condition, and the real-time position information of the vehicle in the traffic scene is not determined based on the matching result.

In some examples, after the processor 401 executes the step of determining that the matching result does not satisfy a preset information matching condition and that the real-time location information of the vehicle in the traffic scene is not determined according to the matching result, the processor 401 further executes:

The vehicle is controlled to move in the traffic scene to continuously collect observation data of local objects of the traffic scene, and description information is determined and stored using the continuously collected observation data.

In some examples, the merging of the first description information with the second description information is performed under the following conditions:

The matching result does not satisfy a preset information matching condition, and it is determined based on the matching result that the real-time position information of the vehicle in the traffic scene does not satisfy a preset stable state, and the first description information is merged with the second description information.

In some examples, the real-time position information of the vehicle in the traffic scene does not satisfy a preset stable state, including: the real-time position of the vehicle displayed on the user interface drifts.

In some examples, the fusion method includes any of the following:

Merging the first description information into the map data; or,

Part of the second description information in the map data is modified according to the first description information.

In some examples, the sensor collects the observation data according to a preset sampling period;

The description information includes key observation data, where the key observation data represents the observation data collected by the sensor within a target sampling period.

In some examples, the key observation data includes: the observation data collected by the sensor within a target sampling period when a preset change occurs in the vehicle posture.

In some examples, the sensor includes one or more of the following sensors: an image sensor, a lidar sensor, a millimeter wave radar sensor, or an ultrasonic sensor.

In some examples, the key observation data includes one or more of the following:

Image data collected by the image sensor during a target sampling period;

Point cloud data collected by the LiDAR sensor during the target sampling period;

Millimeter wave data collected by the millimeter wave radar sensor during the target sampling period;

Ultrasonic data collected by the ultrasonic sensor during the target sampling period.

In some examples, data features of the observation data collected within the target sampling period meet preset data feature conditions.

In some examples, image data collected by the image sensor within a target sampling period meets a preset image texture condition.

In some examples, point cloud data collected by the lidar sensor within a target sampling period meets preset point cloud distribution conditions.

In some examples, the target sampling period is determined based on the observed data of a plurality of the sampling periods.

In some examples, the description information includes tag information;

The tag information includes one or more of the following:

Label information used to characterize the category to which a specific scene in the traffic scene belongs;

Label information used to characterize the collection time of the observation data of the traffic scene.

In some examples, the tag information also includes:

Label information used to characterize the features of the key observation data.

In some examples, the characteristics of the key observations include visual characteristics of the key observations;

The visual feature includes at least one of the following: a SiFT feature, an ORB feature or an MSER feature.

In some examples, the description information includes tag information and/or key observation data, and the processor 401 performs matching of the first description information and the second description information to obtain a matching result, including:

matching the tag information determined by the currently collected observation data with the tag information in the map data; and/or,

The key observation data determined by the currently collected observation data are matched with the key observation data in the map data.

In some examples, the traffic scene has multiple pieces of map data, each piece of map data has usage attribute information, and the usage attribute information is determined based on the tag information;

The processor 401 executes the step of acquiring the map data of the traffic scene, including:

Displaying usage attribute information of the plurality of map data on a user interface, and obtaining usage attribute information selected by a user through the user interface;

The map data is determined from the plurality of map data according to the usage attribute information selected by the user.

In some examples, the traffic scene has multiple pieces of map data, each piece of map data has usage attribute information, and the usage attribute information is determined based on the tag information;

The processor 401 executes the step of acquiring the map data of the traffic scene, including:

The tag information in the first description information is matched with the usage attribute information of the plurality of map data, and the map data is determined from the plurality of map data according to the matching result.

In some examples, the processor 401 further executes:

Send updated map data to the server and/or other vehicles.

As shown in FIG5 , it is a hardware structure diagram of a map data construction device 500 provided in this embodiment. The map data construction device includes a processor 501, a memory 502, and a computer program stored in the memory and executable by the processor. When the processor 501 executes the computer program, the following steps are implemented:

Controlling the vehicle to move in a traffic scene, and obtaining observation data of the global scene of the traffic scene collected by the vehicle's sensor;

Determine description information of the global scene of the traffic scene based on the observation data;

The description information of the global scene is stored as map data of the traffic scene; the description information is used to determine the position information of the vehicle in the traffic scene and whether the map data needs to be updated when the vehicle enters the traffic scene again.

In some examples, the sensor collects the observation data according to a preset sampling period;

The description information includes key observation data, where the key observation data represents the observation data collected by the sensor within a target sampling period.

In some examples, the key observation data includes: the observation data collected by the sensor within a target sampling period when a preset change occurs in the vehicle posture.

In some examples, the sensor includes one or more of the following sensors: an image sensor, a lidar sensor, a millimeter wave radar sensor, or an ultrasonic sensor.

In some examples, the key observation data includes one or more of the following:

Image data collected by the image sensor during a target sampling period;

Point cloud data collected by the LiDAR sensor during the target sampling period;

Millimeter wave data collected by the millimeter wave radar sensor during the target sampling period;

Ultrasonic data collected by the ultrasonic sensor during the target sampling period.

In some examples, data features of the observation data collected within the target sampling period meet preset data feature conditions.

In some examples, image data collected by the image sensor within a target sampling period meets a preset image texture condition.

In some examples, point cloud data collected by the lidar sensor within a target sampling period meets preset point cloud distribution conditions.

In some examples, the target sampling period is determined based on the observed data of a plurality of the sampling periods.

In some examples, the description information includes tag information;

The tag information includes one or more of the following:

Label information used to characterize the category to which a specific scene in the traffic scene belongs;

Label information used to characterize the collection time of the observation data of the traffic scene.

In some examples, the tag information also includes:

Label information used to characterize the features of the key observation data.

In some examples, the characteristics of the key observations include visual characteristics of the key observations;

The visual feature includes at least one of the following: a SiFT feature, an ORB feature or an MSER feature.

In some examples, the processor 501 further executes: sending the map data to a server and/or other vehicles.

As shown in FIG. 6 , the embodiment of the present application further provides a vehicle 600 , including: one or more sensors 610 ; and a map data processing device 400 and/or a map data constructing device 500 .

An embodiment of the present application further provides a computer-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed, the steps of the map data processing method described in any embodiment are implemented.

The embodiment of the present application further provides a computer-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed, the steps of the map data construction method described in any embodiment are implemented.

The embodiments of this specification may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

The method and device provided in the embodiments of the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims (42)

1. A map data processing method, characterized in that the method comprises:

   Acquiring observation data of a local scene of a traffic scene currently acquired by a sensor of a vehicle;

   Determining first descriptive information of the local scene based on the observation data;

   acquiring map data of the traffic scene, wherein the map data comprises second description information generated based on historical observation data of a global scene of the traffic scene;

   Matching the first description information with the second description information to obtain a matching result;

   And updating the map data according to the matching result and the observed data of the currently acquired traffic scene.
2. The method according to claim 1, wherein the method further comprises:

   And if the matching result meets a preset information matching condition, determining real-time position information of the vehicle in the traffic scene according to the matching result, and controlling the vehicle to move in the traffic scene based on the real-time position information.
3. The method of claim 1, wherein the updating the map data based on the matching result and the observed data of the currently collected traffic scene comprises:

   And if the matching result does not meet the preset information matching condition, updating the map data according to the first description information.
4. A method according to claim 1 or 3, wherein the update mode of the map data comprises any one of the following:

   replacing the first descriptive information with the second descriptive information; or alternatively, the first and second heat exchangers may be,

   And fusing the first descriptive information with the second descriptive information.
5. The method of claim 4, wherein said replacing the first description information with the second description information is performed under the following conditions:

   And the matching result does not meet the preset information matching condition, and the real-time position information of the vehicle in the traffic scene is not determined according to the matching result.
6. The method according to claim 5, wherein after the step of determining that the matching result does not satisfy a preset information matching condition and that the real-time position information of the vehicle in the traffic scene is not determined according to the matching result, the method further comprises:

   And controlling the vehicle to move in the traffic scene so as to continuously collect the observation data of the local scenery of the traffic scene, and determining and storing description information by utilizing the continuously collected observation data.
7. The method of claim 4, wherein the fusing the first description information with the second description information is performed under the following conditions:

   and the matching result does not meet the preset information matching condition, and according to the matching result, the real-time position information of the vehicle in the traffic scene is determined to not meet the preset stable state, and the first description information and the second description information are fused.
8. The method of claim 7, wherein the real-time location information of the vehicle in the traffic scene does not satisfy a preset steady state, comprising: the real-time position of the vehicle displayed on the user interface shifts.
9. The method of claim 4, wherein the means of fusing comprises any one of:

   incorporating the first description information into the map data; or alternatively, the first and second heat exchangers may be,

   And modifying part of the second descriptive information in the map data according to the first descriptive information.
10. The method of claim 1, wherein the sensor collects the observation data at a preset sampling period;

    the descriptive information includes critical observations characterizing the observations acquired by the sensor over a target sampling period.
11. The method of claim 10, wherein the critical observation data comprises: when the vehicle posture is subjected to preset change, the sensor acquires the observation data in a target sampling period.
12. The method of claim 10, wherein the sensor comprises one or more of the following: an image sensor, a lidar sensor, a millimeter wave radar sensor or an ultrasonic sensor.
13. The method of claim 12, wherein the critical observation data comprises one or more of:

    image data acquired by the image sensor during a target sampling period;

    Point cloud data acquired by a lidar sensor within a target sampling period;

    millimeter wave data acquired by a millimeter wave radar sensor within a target sampling period;

    ultrasonic data acquired by the ultrasonic sensor during a target sampling period.
14. The method of claim 13, wherein the data characteristic of the observed data acquired during the target sampling period meets a preset data characteristic condition.
15. The method of claim 14, wherein the image data acquired by the image sensor during the target sampling period satisfies a predetermined image texture condition.
16. The method of claim 14, wherein the point cloud data collected by the lidar sensor during the target sampling period satisfies a predetermined point cloud distribution condition.
17. The method of claim 14, wherein the target sampling period is determined from the observed data for a plurality of the sampling periods.
18. The method according to claim 1 or 10, wherein the description information comprises tag information;

    the tag information includes one or more of the following:

    Tag information for characterizing a category to which a particular scene belongs in the traffic scene;

    Tag information characterizing the acquisition time of the observation data of the traffic scene.
19. The method of claim 18, wherein the tag information further comprises:

    tag information characterizing the key observations.
20. The method of claim 19, wherein the characteristic of the critical observation comprises a visual characteristic of the critical observation;

    The visual features include at least one of: siFT, ORB, or MSER features.
21. The method according to claim 1 or 18, wherein the description information includes tag information and/or key observation data, and the matching the first description information and the second description information to obtain a matching result includes:

    matching the tag information determined by the currently acquired observation data with the tag information in the map data; and/or the number of the groups of groups,

    And matching the key observation data determined by the currently acquired observation data with the key observation data in the map data.
22. The method of claim 18, wherein the traffic scene has a plurality of map data, each map data having usage attribute information, the usage attribute information being determined based on the tag information;

    the obtaining the map data of the traffic scene comprises the following steps:

    Displaying the usage attribute information of the map data on a user interface, and acquiring the usage attribute information selected by a user through the user interface;

    And determining the map data from the map data according to the use attribute information selected by the user.
23. The method of claim 18, wherein the traffic scene has a plurality of map data, each map data having usage attribute information, the usage attribute information being determined based on the tag information;

    the obtaining the map data of the traffic scene comprises the following steps:

    and matching the tag information in the first description information with the use attribute information of the map data, and determining the map data from the map data according to the matching result.
24. The method according to claim 1, wherein the method further comprises:

    And sending the updated map data to a server and/or other vehicles.
25. A map data construction method, characterized in that the method comprises:

    Controlling a vehicle to move in a traffic scene, and acquiring observation data of a global scene of the traffic scene, which is acquired by a sensor of the vehicle;

    determining the description information of the global scene of the traffic scene based on the observation data;

    storing the description information of the global scene as map data of the traffic scene; and the description information is used for determining the position information of the vehicle in the traffic scene and determining whether the map data needs to be updated when the vehicle reenters the traffic scene.
26. The method of claim 25, wherein the sensor collects the observation data at a preset sampling period;

    the descriptive information includes critical observations characterizing the observations acquired by the sensor over a target sampling period.
27. The method of claim 26, wherein the critical observation data comprises: when the vehicle posture is subjected to preset change, the sensor acquires the observation data in a target sampling period.
28. The method of claim 26, wherein the sensor comprises one or more of the following: an image sensor, a lidar sensor, a millimeter wave radar sensor or an ultrasonic sensor.
29. The method of claim 28, wherein the critical observation data comprises one or more of:

    image data acquired by the image sensor during a target sampling period;

    Point cloud data acquired by a lidar sensor within a target sampling period;

    millimeter wave data acquired by a millimeter wave radar sensor within a target sampling period;

    ultrasonic data acquired by the ultrasonic sensor during a target sampling period.
30. The method of claim 25, wherein the data characteristic of the observed data acquired during the target sampling period meets a preset data characteristic condition.
31. The method of claim 30, wherein the image data acquired by the image sensor during the target sampling period satisfies a predetermined image texture condition.
32. The method of claim 30, wherein the point cloud data collected by the lidar sensor during the target sampling period satisfies a predetermined point cloud distribution condition.
33. The method of claim 30, wherein the target sampling period is determined from the observed data for a plurality of the sampling periods.
34. The method of claim 25 or 26, wherein the descriptive information comprises tag information;

    the tag information includes one or more of the following:

    Tag information for characterizing a category to which a particular scene belongs in the traffic scene;

    Tag information characterizing the acquisition time of the observation data of the traffic scene.
35. The method of claim 34, wherein the tag information further comprises:

    tag information characterizing the key observations.
36. The method of claim 35, wherein the characteristic of the critical observation comprises a visual characteristic of the critical observation;

    The visual features include at least one of: siFT, ORB, or MSER features.
37. The method of claim 25, wherein the method further comprises:

    And sending the map data to a server and/or other vehicles.
38. A map data processing device, characterized in that the device comprises a processor, a memory, a computer program stored on the memory that is executable by the processor, the processor implementing the method of any one of claims 1 to 24 when executing the computer program.
39. A map data construction device, characterized in that the device comprises a processor, a memory, a computer program stored on the memory that is executable by the processor, the processor implementing the method of any of claims 25 to 37 when executing the computer program.
40. A vehicle, characterized in that the vehicle comprises:

    One or more sensors; and

    The map data processing device of claim 38, and/or the map data construction device of claim 39.
41. A computer readable storage medium having stored thereon computer instructions which when executed perform the steps of the map data processing method of any of claims 1 to 24.
42. A computer readable storage medium having stored thereon computer instructions which when executed perform the steps of the map data construction method of any of claims 25 to 37.