CN108280886A - Laser point cloud mask method, device and readable storage medium storing program for executing - Google Patents

Abstract

Embodiments of the present invention provide a laser point cloud labeling method, device and readable storage medium, which belong to the field of recognition technology. The method includes: obtaining at least two frames of laser point cloud data including the first frame of laser point cloud data and the second frame of laser point cloud data; superimposing and displaying the first object point cloud data and the first object point cloud data of the first object in the three-dimensional scene The second object point cloud data of the second object; create a first frame for the first object point cloud data and create a second frame for the second object point cloud data; use the first frame for The point cloud data of the first object is marked with a first mark, and the point cloud data of the second object is marked with a second mark by using the second check box. Through this method, multiple frames of laser point cloud data can be marked simultaneously, without needing to mark all the laser point cloud data one by one, thereby improving the speed and efficiency of marking laser points.

Description

Laser point cloud labeling method, device and readable storage medium

technical field

The present invention relates to the field of recognition technology, in particular to a laser point cloud marking method, device and readable storage medium.

Background technique

At present, with the development of automatic driving technology, it is particularly important to identify target objects around the vehicle (such as vehicles, pedestrians, tricycles, bicycles, etc.). line or 64-line laser radar) to detect target objects around the vehicle, the laser radar emits laser beams around, and when the laser beam encounters an object, it returns a laser point cloud, and identifies the surrounding target objects and the size of the target object through the laser point cloud , position, speed, etc.

At present, the main way to identify the target object through the laser point cloud is: manually mark the received laser point cloud point by point in advance to obtain the laser point cloud sample data corresponding to the target object; use the sample data for machine learning to obtain object recognition Model; through the object recognition model, the target object corresponding to the laser point cloud is recognized.

At present, the received laser point cloud is manually marked point by point, and the laser point cloud contains a huge amount of laser point data. This marking method is slow, and the laser point cloud also contains a large number of laser points that are not target objects. Annotators may also process a large number of laser points of non-target objects, which is extremely inefficient.

Contents of the invention

In view of this, the purpose of the embodiments of the present invention is to provide a laser point cloud marking method, device and readable storage medium, so as to improve the above problems.

In the first aspect, an embodiment of the present invention provides a laser point cloud labeling method, the method comprising: obtaining at least two frames of laser point cloud data including the first frame of laser point cloud data and the second frame of laser point cloud data, wherein , the first frame of laser point cloud data includes at least the first object point cloud data of the first object, and the second frame of laser point cloud data includes at least the second object point cloud data of the second object; in a three-dimensional scene The first object point cloud data of the first object and the second object point cloud data of the second object are displayed in an inner superimposition; a first check box is created for the first object point cloud data and a first frame is created for the second object Create a second frame for the point cloud data; use the first frame to mark the first mark for the point cloud data of the first object, and use the second frame to mark the second mark for the point cloud data of the second object mark.

Further, when the first object and the second object are different objects, the first mark and the second mark are different; when the first object and the second object are the same object , the first label is the same as the second label.

Further, the first mark includes a first number or includes the first number and the first type of the first object; the second mark includes a second number or includes the second number and the first type Second type of object.

Further, creating a first frame for the point cloud data of the first object and creating a second frame for the point cloud data of the second object includes: obtaining a frame creation request input by a user; creating a frame based on the frame Request to create a first frame for the point cloud data of the first object and create a second frame for the point cloud data of the second object.

Further, creating a first frame for the point cloud data of the first object and creating a second frame for the point cloud data of the second object based on the frame creation request includes: creating a frame based on the frame creation request The first object point cloud data and the second object point cloud data create an object frame; the object frame is divided into the first object point cloud data corresponding to the first frame and the second object point The second check box corresponding to cloud data.

Further, superimposing and displaying the first object point cloud data of the first object and the second object point cloud data of the second object in the three-dimensional scene includes: based on the first object point cloud data and the second object point cloud data The two-object point cloud data constructs a three-dimensional scene, and establishes a three-dimensional coordinate system corresponding to the three-dimensional scene; the coordinates of each laser point in the first object point cloud data and the second object point cloud data are converted into the The three-dimensional coordinates in the three-dimensional coordinate system; according to the three-dimensional coordinates of each laser point, each laser point is placed in the three-dimensional scene for display.

In a second aspect, an embodiment of the present invention provides a laser point cloud labeling device, the device comprising: a point cloud data acquisition module, used to obtain at least one frame of laser point cloud data and a second frame of laser point cloud data Two frames of laser point cloud data, wherein the first frame of laser point cloud data contains at least the first object point cloud data of the first object, and the second frame of laser point cloud data contains at least the second object point cloud data of the second object. Object point cloud data; display module, used to overlay and display the first object point cloud data of the first object and the second object point cloud data of the second object in the three-dimensional scene; frame creation module, used for The first object point cloud data creates a first frame and creates a second frame for the second object point cloud data; a marking module is used to use the first frame to create a second frame for the first object point cloud data mark a first mark, and use the second check box to mark a second mark for the point cloud data of the second object.

Further, the marquee creation module includes: a request acquisition unit, configured to obtain a marquee creation request input by a user; a creation unit, configured to create a second object point cloud data for the first object based on the marquee creation request. A selection box and creating a second selection box for the point cloud data of the second object.

Further, the display module includes: a three-dimensional scene establishment unit, configured to construct a three-dimensional scene based on the point cloud data of the first object and the point cloud data of the second object, and establish three-dimensional coordinates corresponding to the three-dimensional scene system; a coordinate conversion unit for converting the coordinates of each laser point in the first object point cloud data and the second object point cloud data into three-dimensional coordinates in the three-dimensional coordinate system; a display unit for Put each laser point into the three-dimensional scene for display according to the three-dimensional coordinates of each laser point.

In a third aspect, an embodiment of the present invention provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

The beneficial effects of the embodiments of the present invention are:

Embodiments of the present invention provide a laser point cloud labeling method, device, and readable storage medium. The method first obtains at least two frames of laser point cloud data including the first frame of laser point cloud data and the second frame of laser point cloud data, wherein , the first frame of laser point cloud data contains at least the first object point cloud data of the first object, the second frame of laser point cloud data contains at least the second object point cloud data of the second object, and then in three-dimensional In the scene, the first object point cloud data of the first object and the second object point cloud data of the second object are superimposed and displayed, and then a first frame is created for the first object point cloud data and a frame is created for the second object. Create a second frame for the point cloud data of the two objects, then use the first frame to mark the first mark for the point cloud data of the first object, and use the second frame for the point cloud data of the second object Marking the second mark, through this method, multiple frames of laser point cloud data can be marked simultaneously without marking all the laser point cloud data one by one, thereby improving the marking speed and efficiency of laser points.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

FIG. 1 shows a structural block diagram of an electronic device applicable to an embodiment of the present application;

Fig. 2 is a flowchart of a laser point cloud labeling method provided by an embodiment of the present invention;

Fig. 3 is a schematic diagram of a laser point cloud data display in a three-dimensional scene provided by an embodiment of the present invention;

FIG. 4 is a flowchart of step S120 in a laser point cloud labeling method provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram showing another laser point cloud data in a three-dimensional scene provided by an embodiment of the present invention;

Fig. 6 is a structural block diagram of a laser point cloud labeling device provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. The components of the embodiments of the invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present invention.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

Please refer to FIG. 1 , which shows a structural block diagram of an electronic device 100 applicable to an embodiment of the present application. The electronic device 100 can be a terminal device, which includes a laser point cloud labeling device, a memory 101 , a storage controller 102 , a processor 103 , a peripheral interface 104 , an input and output unit 105 , an audio unit 106 , and a display unit 107 .

The memory 101, storage controller 102, processor 103, peripheral interface 104, input and output unit 105, audio unit 106, and display unit 107 are electrically connected to each other directly or indirectly to realize data transmission or interact. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The laser point cloud labeling device includes at least one software function module that can be stored in the memory 101 in the form of software or firmware (firmware) or solidified in the operating system (operating system, OS) of the laser point cloud labeling device . The processor 103 is configured to execute executable modules stored in the memory 101 , such as software function modules or computer programs included in the laser point cloud labeling device.

Wherein, memory 101 can be, but not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-OnlyMemory, PROM), erasable In addition to read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc. Wherein, the memory 101 is used to store the program, and the processor 103 executes the program after receiving the execution instruction, and the method executed by the server of the stream process definition disclosed in any of the embodiments of the present invention can be applied to process In the device 103, or implemented by the processor 103.

The processor 103 may be an integrated circuit chip, which has a signal processing capability. Above-mentioned processor 103 can be general-purpose processor, comprises central processing unit (Central Processing Unit, be called for short CPU), network processor (Network Processor, be called for short NP) etc.; Can also be digital signal processor (DSP), application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor, or the processor 103 may be any conventional processor or the like.

The peripheral interface 104 couples various input/output devices to the processor 103 and the memory 101 . In some embodiments, peripheral interface 104, processor 103, and memory controller 102 may be implemented in a single chip. In some other instances, they can be implemented by independent chips respectively.

The input and output unit 105 is used to provide the user with input data to realize the interaction between the user and the server (or local terminal). The input and output unit 105 may be, but not limited to, a mouse and a keyboard.

The audio unit 106 provides an audio interface to the user and may include one or more microphones, one or more speakers, and audio circuitry.

The display unit 107 provides an interactive interface (such as a user operation interface) between the electronic device 100 and the user or is used to display image data for the user's reference. In this embodiment, the display unit 107 may be a liquid crystal display or a touch display. If it is a touch display, it can be a capacitive touch screen or a resistive touch screen supporting single-point and multi-touch operations. Supporting single-point and multi-touch operations means that the touch display can sense simultaneous touch operations from one or more positions on the touch display, and hand over the sensed touch operations to the processor 103 Perform calculations and processing.

The peripheral interface 104 couples various input/output devices to the processor 103 and the memory 101 . In some embodiments, peripheral interface 104, processor 103, and memory controller 102 may be implemented in a single chip. In some other instances, they can be implemented by independent chips respectively.

The input and output unit 105 is used to provide the user with input data to realize the interaction between the user and the processing terminal. The input and output unit 105 may be, but not limited to, a mouse and a keyboard.

It can be understood that the structure shown in FIG. 1 is only for illustration, and the electronic device 100 may also include more or less components than those shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 . Each component shown in Fig. 1 may be implemented by hardware, software or a combination thereof.

Please refer to Fig. 2, Fig. 2 is a flow chart of a laser point cloud labeling method provided by an embodiment of the present invention, the method includes the following steps:

Step S110: Obtain at least two frames of laser point cloud data including the first frame of laser point cloud data and the second frame of laser point cloud data.

Laser point cloud data refers to the use of lasers to obtain the spatial coordinates of each sampling point on the surface of an object in the same spatial reference system, and obtain a series of massive point collections that express the spatial distribution of the target and the characteristics of the target surface. This point set is called It is called "point cloud". The properties of point cloud include spatial resolution, point accuracy, surface normal vector, etc.

In the field of autonomous driving, in order to identify target objects (such as vehicles, pedestrians, tricycles, bicycles, etc.) It can automatically identify the surrounding objects automatically.

In the process of obtaining data samples of the target object, in order to improve the efficiency of sample acquisition, it is first necessary to obtain multiple frames of laser point cloud data. Specifically, the multi-frame laser point cloud data includes the first frame of laser point cloud data and the second frame of At least two frames of laser point cloud data of laser point cloud data, wherein the first frame of laser point cloud data contains at least the first object point cloud data of the first object, and the second frame of laser point cloud data contains at least Second object point cloud data of the second object.

Specifically, as shown in FIG. 3 , laser point cloud data can be collected by a laser point cloud device, which can be integrated into a worker's backpack or a mobile collection platform. For example, when it is necessary to collect laser point cloud data of target objects on the road, the laser point cloud device traverses the entire scene within a specific range as the staff moves, and during the above process, the laser point cloud device Set the time to collect the laser point cloud data in the whole scene, and the collection time interval of each frame of laser point cloud data is the preset time interval (such as 0.1s), so that multiple frames of laser point cloud data can be obtained. Each frame of laser point cloud data is relative to the spatial coordinate system of the laser point cloud device at the time of the laser point cloud data collection, and the spatial coordinate system of the laser point cloud device is different at different collection times. Of course, the pre-acquired multi-frame laser point cloud data can be stored in the server, and can be called directly when needed later.

In addition, as a way, during the driving process of the vehicle, the on-board laser scanner on the vehicle can also be used to collect multi-frame laser point clouds of each target object on the road where the vehicle is driving at a preset acquisition frequency (such as an interval of 0.1s) , and then send the collected multi-frame laser point cloud data to the server for storage. When the target object needs to be marked, the laser point cloud data of the target object can be obtained from the server and displayed in the 3D scene for marking.

Among them, the first frame of laser point cloud data can be the laser point cloud data collected by the laser point cloud device at the first moment, and the second frame of laser point cloud data can be the laser point cloud data collected by the laser point cloud device at the second moment, Wherein, the interval between the first moment and the second moment can be 0.1s, and of course, multiple frames of laser point cloud data such as the third frame and the fourth frame can be continuously collected. Wherein, the first object and the second object can be different target objects, then the first object point cloud data and the second object point cloud data are laser point cloud data corresponding to different target objects, of course the first object and the second object It can also be the same target object, then the point cloud data of the first object and the point cloud data of the second object are the laser point cloud data of the first object and the second object collected at different times.

Step S120: Superimposing and displaying the first object point cloud data of the first object and the second object point cloud data of the second object in the three-dimensional scene.

In order to mark the target object, it is necessary to construct a 3D scene of the target object. It is necessary to reposition the laser point cloud data in different spatial coordinate systems to generate a 3D map in a unified coordinate system.

Specifically, referring to FIG. 4, step S120 includes:

Step S121: Construct a 3D scene based on the point cloud data of the first object and the point cloud data of the second object, and establish a 3D coordinate system corresponding to the 3D scene.

3D scene creation refers to the 3D visualization of the point cloud data of the target object under the terminal device, which can be created using related tools, such as Arcgis tools.

Of course, in the embodiment of the present invention, the 3D scene can be constructed by but not limited to using WebGL technology, OSG (OpenSceneGraph) or STK (Satellite Tool Kit), and the method of constructing the 3D scene is not strictly limited in this application.

Step S122: Convert the coordinates of each laser point in the first object point cloud data and the second object point cloud data into three-dimensional coordinates in the three-dimensional coordinate system.

The collection of laser point cloud data is carried out in unit time, so the coordinate system of each frame of laser point cloud data is independent. Therefore, it is necessary to register each laser point cloud data and normalize each laser point cloud data to a unified In the coordinate system, the coordinates of each laser point in the first object point cloud data and the second object point cloud data are converted into three-dimensional coordinates in the three-dimensional coordinate system. Among them, the registration of laser point cloud data can be completed by Cyclone software tool.

Of course, the pairwise registration between point cloud data is actually to unify the point cloud data in two coordinate systems into the same coordinate system through coordinate transformation. The conversion relationship between point clouds includes 3 rotation parameters and 3 For the convenience of calculation, the three rotation parameters are usually expressed as a 3*3 rotation matrix R, and the three translation parameters are expressed as a three-dimensional translation vector T. The purpose of the point cloud configuration is to find out the rigid body transformation that satisfies the conditions ( R, T), the registration method can adopt the point cloud registration method based on discrete features (points, lines, surfaces) and the iterative closest point, that is, the ICP registration method, and will not go into details here.

Step S123: putting each laser point into the three-dimensional scene for display according to the three-dimensional coordinates of each laser point.

Put the first object point cloud data and the second object point cloud data into the three-dimensional scene according to the three-dimensional coordinates of each laser point for display, as shown in Figure 3, it can see the points of each target object Presentation of cloud data in a 3D scene. It can also be seen from Figure 3 that the trajectory of the same object in the 3D scene generated under the multi-frame laser point cloud data.

Step S130: Create a first frame for the point cloud data of the first object and create a second frame for the point cloud data of the second object.

After checking the point cloud data of the first object and presenting the second point cloud data in the 3D scene, the corresponding point cloud data of each target object in the 3D scene can be marked.

Putting the laser point cloud into the 3D scene, since the laser points that belong to the same target object are relatively concentrated and can display the general outline of the target object, the labeling personnel can more intuitively and quickly judge the laser points that belong to the target type, thus The laser points belonging to the target type can be marked out quickly and in a targeted manner. It is not necessary to mark out all the laser points one by one before marking the laser points belonging to the target type, thereby improving the speed and efficiency of laser point marking.

In practical application, the number of laser points belonging to the same target object is relatively large and relatively concentrated. Slower, therefore, in order to further improve the speed of laser point marking, the technical solution of the present invention is to uniformly mark multiple laser points of the same type once or more times, that is, generate a three-dimensional marquee according to the laser points that need to be marked, by having A three-dimensional marquee in a certain three-dimensional space is used to select multiple laser points that need to be marked uniformly, and the laser points that fall into the three-dimensional marquee are uniformly marked.

For example, when it is necessary to create a first box for the point cloud data of the first object and a second box for the point cloud data of the second object, the labeler can input a box creation request according to his own labeling requirements, and the terminal device can obtain Based on the frame creation request input by the user, a first frame is created for the point cloud data of the first object and a second frame is created for the point cloud data of the second object based on the frame creation request.

Specifically, the annotator can select the point cloud data of the first object to be annotated with the mouse, and then input corresponding instructions on the interface of the 3D scene displayed on the terminal device, such as the size information of the first selection box. If the first selection box defaults to If it is a three-dimensional rectangular frame, you can enter the corresponding size information such as length, width, and height. After obtaining the confirmation command clicked by the user, you can create the first frame, and the laser point cloud data in the first frame is the first object. Point cloud data, because in the display interface, the first frame can only be seen as a two-dimensional plane graphic, as shown in Figure 5; if the first frame is a three-dimensional polygon frame, the user can display the three-dimensional image on the terminal device In the interface of the scene, click with the mouse, that is, one point of the first selection frame is clicked once with the mouse, and another point of the first selection frame is clicked for the second time, and the two points are connected. After multiple clicks of the mouse , by obtaining the confirmation instruction clicked by the user, a closed figure of the first selection box can be formed, thereby creating a polygonal first selection box. Similarly, the creation of the second selection box can also be carried out through the above method, which will not be repeated here.

Of course, in addition to creating the first frame for the point cloud data of the first object and the second frame for the point cloud data of the second object, it is also possible to create a frame for the point cloud data of the first object and the second frame based on the frame creation request. An object frame is created for the object point cloud data, and the object frame is divided into a first frame corresponding to the first object point cloud data and a second frame corresponding to the second object point cloud data.

Specifically, the user can select the point cloud data of the first object and the point cloud data of the second object to be marked with the mouse, and then input corresponding instructions on the interface of the three-dimensional scene displayed on the terminal device, such as the size information of the object frame, or Select the size of the first selection box in the drop-down option of the input box on the interface, or set the size of the first selection box in the preset size bar. If the object selection box defaults to a three-dimensional rectangle selection box, you can enter the corresponding For size information such as length, width and height, after obtaining the confirmation instruction clicked by the user, an object frame can be created, and the laser point cloud data in the object frame includes the point cloud data of the first object and the point cloud data of the second object. Of course, For the method of creating a polygonal object selection frame, please refer to the process described above, and details will not be repeated here. If the first object and the second object are different objects, the first object and the second object need to be marked separately. Since there may be a distance between the laser point clouds of the first object and the second object, segmentation can be used The algorithm separates the point cloud data of the first object from the point cloud data of the second object. The segmentation algorithm can use region-based segmentation algorithms, such as watershed, region merging and splitting, etc., or edge-based segmentation algorithms, such as Lapp Las operator, etc., after the first object point cloud data and the second object point cloud data can be segmented through the segmentation algorithm, the first frame can be created for the first object point cloud data, and the second point cloud data can be created for the second object. A box, of course, the first box contains the point cloud data of the first object, and the second box contains the point cloud data of the second object, and the shapes of the first box and the second box are not limited.

Step S140: Use the first check box to mark the first object point cloud data with a first mark, and use the second select box to mark the second object point cloud data with a second mark.

Due to the large number of laser points contained in the laser point cloud, in order to avoid missing labels and improve the comprehensiveness and integrity of laser point labeling, in the embodiment of the present invention, after creating a 3D scene, a camera is built in the 3D scene, and by adjusting the camera View the laser point in the 3D scene at the position and direction in the 3D scene to ensure that the labeler can view the laser point in 360 degrees in the 3D scene.

As an implementation manner, at least one camera can be constructed in the three-dimensional scene, so as to adjust the viewing angle by adjusting the position and direction of the camera in the three-dimensional scene during the process of marking the laser point in the three-dimensional scene. Viewing angle and extent of laser point cloud in 3D scene.

In the embodiment of the present invention, the laser point cloud labeling method is applicable to the browser. When the browser receives the laser point cloud data sent by the server, it uses the aforementioned laser point cloud labeling method to label the received laser point cloud, and Feedback the annotation result to the server.

In order to further improve the accuracy of the marking personnel in determining the type of laser point in the 3D scene, in the technical solution of the present invention, at least two cameras are constructed in the 3D scene. At the same time, the positions and directions of each camera are different, so that the marking personnel can The laser point can be viewed from different viewing angles, which improves the accuracy of judging the type of the laser point.

Preferably, the aforementioned at least two cameras can be switched according to the user's selection. When the user selects one of the cameras, the corresponding viewing angle and range of the three-dimensional scene are presented to the user with the position and direction of the selected camera, and the unselected camera is displayed in the position and direction of the selected camera. Thumbnails present the corresponding perspective and range of the 3D scene.

As a result, the first object and the second object can be identified more accurately under a suitable viewing angle, so as to mark the first object and the second object, then the mark can be created after the first frame and the second frame are created Input the corresponding labeling information. When the first object and the second object are different objects, the first mark and the second mark are different. For example, if the first object is a car, the first The mark can be set to a, if the second object is a large truck, then the second mark can be set to b, when the first object and the second object are the same object, the first mark and the second The marks are the same, for example, when the first object and the second object are both cars, the first mark and the second mark are both a, that is, the terminal device is pre-stored with marks corresponding to various types of target objects, such as cars Corresponding to a, large trucks corresponding to b, and pedestrians corresponding to c, after marking the first object and the second object, reliable data can be provided in the subsequent object recognition process.

Among them, the identification of the target object can be judged by the labeler based on experience. For example, if the first object is a car according to the observation, the labeler can input the labeling information of the first mark as a, and the second object is a large truck. When , the labeler can input the labeling information of the second mark b, thereby completing the labeling of the target object. Of course, the identification of the target object can be carried out by the terminal device. For example, the terminal device has pre-stored contours of various target objects, and the point cloud data of the target object can form the contour map and size of the target object in the three-dimensional scene. For example, when the first object is a car, the point cloud data of the first object can be formed into the outline and size of the car, and the terminal device can combine the point cloud data of the first object with the outline of the stored target object Compare images and sizes. When the similarity exceeds a certain threshold, such as 90%, it is judged to be the same object, and the first object can be automatically marked. For example, the first mark is marked as a, thereby reducing the work of the labeling personnel Of course, if the labeler judges based on experience that the identification of the terminal device is wrong, he can modify the labeling information by himself.

Wherein, the first mark contains a first number or contains the first number and the first type of the first object; the second mark contains a second number or contains the second number and the second The second type of object. For example, for the first object is a car, its first mark is a, and the information it includes can be the first number 1, or the first number 1 and the first type (car) of the first object, for the first number 1 The second object is a large truck, its second mark is b, and the information it includes can be the second number 2, or the second number 2 and the second type of the second object (big truck).

In addition, according to the different types of the first object and the second object, you can also select different labeling styles for labeling, for example, set the color of the created first box and the color of the second box to be different, or set the first The color of the point cloud data of the first object in the selection box and the point cloud data of the second object in the second selection box are set to be different. For example, the point cloud of the first object is a car and its point cloud can be marked as blue after selection , the second object is a large truck, and its point cloud can be marked in red after being selected, and different colors can be marked when marking according to different types, so that different objects can be distinguished according to the color.

Of course, in order to obtain more information about the target object to accurately identify the target object, the marking of the first object and the second object may also include more annotation information, such as the size and position of the first object , angle and other annotation information.

In addition, as an implementation mode, after multiple target objects can be marked by the above method, the marked target objects can be used as laser point cloud sample data for identifying the target object. When identifying the target object, first use the sample data The data trains a machine learning model to identify the target object, and the machine learning model is a model for identifying the target object based on the laser point cloud sample data.

For example, the label information (such as type, size, position, angle, etc.) of the target object is converted into the input vector of the machine learning model, so as to use the training samples to train the machine learning model. The machine learning model may be a machine learning model that uses the laser point cloud of the target object as input to identify the size, position, and angle of the target object, such as a deep learning model.

In this embodiment, based on the laser point cloud of the target object, the type, size, position, angle and other labeling information of different types of target objects are obtained, so as to accurately determine the type, size, position, and angle of different types of target objects. angle. Then, based on the labeling information of the target object, a training sample of a machine learning model that takes the laser point cloud of the target object as input and recognizes the type, size, position, and angle of the target object can be generated. This training sample can be used to train the machine learning model that recognizes the type, size, position, and angle of the target object by using the laser point cloud of the target object as input, thereby continuously improving the recognition accuracy of the machine learning model.

In some optional implementations of this embodiment, generating the training samples of the machine learning model based on the label information includes: sending the label information to the server, so that the server can generate a machine learning model set on the server based on the label information. The training samples on which the model is trained.

In this embodiment, the laser point cloud of the target object is used as an input, and the machine learning model for identifying the type, size, position, and angle of the target object can be set on the server. The labeling information can be sent to the server, so that training samples can be generated on the server based on the labeling information of the target object, and the type, size, position, angle, and The recognition machine learning model is trained to continuously improve the recognition accuracy of the machine learning model for the target object.

Please refer to FIG. 6. FIG. 6 is a structural block diagram of a laser point cloud labeling device 200 provided by an embodiment of the present invention. The device includes:

The point cloud data acquisition module 210 is used to obtain at least two frames of laser point cloud data including the first frame of laser point cloud data and the second frame of laser point cloud data, wherein the first frame of laser point cloud data contains at least the first frame of laser point cloud data The first object point cloud data of an object, the second frame of laser point cloud data at least includes the second object point cloud data of the second object.

The display module 220 is configured to superimpose and display the point cloud data of the first object of the first object and the point cloud data of the second object of the second object in a three-dimensional scene.

A frame creating module 230, configured to create a first frame for the point cloud data of the first object and a second frame for the point cloud data of the second object.

The marking module 240 is configured to use the first check box to mark the first object point cloud data with a first mark, and use the second select box to mark the second object point cloud data with a second mark.

As a manner, the frame creation module 230 includes:

The request obtaining unit is used for obtaining a selection box creation request input by a user.

A creation unit, configured to create a first frame for the point cloud data of the first object and a second frame for the point cloud data of the second object based on the frame creation request.

As a way, the creation unit includes:

An object frame creation unit, configured to create an object frame based on the frame creation request for the first object point cloud data and the second object point cloud data.

A frame segmentation unit, configured to segment the object frame into a first frame corresponding to the first object point cloud data and a second frame corresponding to the second object point cloud data.

As a way, the display module 220 includes:

A 3D scene establishing unit, configured to construct a 3D scene based on the point cloud data of the first object and the point cloud data of the second object, and establish a 3D coordinate system corresponding to the 3D scene.

A coordinate conversion unit, configured to convert the coordinates of each laser point in the first object point cloud data and the second object point cloud data into three-dimensional coordinates in the three-dimensional coordinate system.

The display unit is configured to display each laser point in the three-dimensional scene according to the three-dimensional coordinates of each laser point.

An embodiment of the present invention also provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above laser point cloud labeling method is implemented.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and details are not repeated here.

In summary, embodiments of the present invention provide a laser point cloud labeling method, device, and readable storage medium. The method first obtains at least two frames of laser point cloud data containing the first frame of laser point cloud data and the second frame of laser point cloud data. Point cloud data, wherein the first frame of laser point cloud data includes at least the first object point cloud data of the first object, and the second frame of laser point cloud data includes at least the second object point cloud of the second object data, and then superimpose and display the first object point cloud data of the first object and the second object point cloud data of the second object in the three-dimensional scene, and then create a first frame for the first object point cloud data And create a second frame for the point cloud data of the second object, then use the first frame to mark the first mark for the point cloud data of the first object, and use the second frame to mark the point cloud data for the first object Two-object point cloud data is marked with a second mark. This method can realize simultaneous marking of multiple frames of laser point cloud data without marking all laser point cloud data one by one, thereby improving the speed and efficiency of laser point marking.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functions and possible implementations of devices, methods and computer program products according to multiple embodiments of the present invention. operate. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

In addition, each functional module in each embodiment of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention. It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Claims (10)

1. a kind of laser point cloud mask method, which is characterized in that the method includes：

At least two frame laser point cloud datas for including first frame laser point cloud data and the second frame laser point cloud data are obtained, In, the first object point cloud data of the first object, the second frame laser are included at least in the first frame laser point cloud data The second object point cloud data of the second object is included at least in point cloud data；

The second couple of the first object point cloud data of the first object described in Overlapping display and second object in three-dimensional scenic As point cloud data；

Frame is selected for the first object-point cloud data creation first and be that the second object-point cloud data creation second selects frame；

It is the first label of the first object point cloud data mark to select frame using described first, and selects frame for institute using described second State the second label of the second object point cloud data mark.

2. according to the method described in claim 1, it is characterized in that, being Bu Tong right in first object and second object As when, it is described first label and it is described second label differs；It is same object in first object and second object When, first label is identical with second label.

3. according to the method described in claim 2, it is characterized in that, first label is comprising the first number or includes described the The first kind of one number and first object；Second label numbers comprising second or includes second number and institute State the Second Type of the second object.

4. according to the method described in any claim in claim 1-3, which is characterized in that created for the first object point cloud data First is built to select frame and select frame for the second object-point cloud data creation second, including：

It obtains and input by user selects frame request to create；

To select frame request to create be that the first object-point cloud data creation first selects frame and is second object-point based on described Cloud data creation second selects frame.

5. according to the method described in claim 4, it is characterized in that, selecting frame request to create for first object-point based on described Cloud data creation first selects frame and selects frame for the second object-point cloud data creation second, including：

The first object point cloud data described in frame request to create and the second object-point cloud data creation object is selected to select based on described Frame；

It selects frame to be divided into the first object point cloud data corresponding first object and selects frame and the second object-point cloud Data corresponding second select frame.

6. according to the method described in any claim in claim 1-3, which is characterized in that in three-dimensional scenic described in Overlapping display First object point cloud data of the first object and the second object point cloud data of second object, including：

Three-dimensional scenic is built based on the first object point cloud data and the second object point cloud data, and is established and described three Tie up the corresponding three-dimensional system of coordinate of scene；

The coordinate of each laser point in the first object point cloud data and the second object point cloud data is converted to described Three-dimensional coordinate in three-dimensional system of coordinate；

Each laser point is put into the three-dimensional scenic according to the three-dimensional coordinate of each laser point and is shown.

7. a kind of laser point cloud annotation equipment, which is characterized in that described device includes：

Point cloud data acquisition module, for obtaining comprising first frame laser point cloud data and the second frame laser point cloud data at least Two frame laser point cloud datas, wherein the first object-point cloud of the first object is included at least in the first frame laser point cloud data Data include at least the second object point cloud data of the second object in the second frame laser point cloud data；

Display module, the first object point cloud data and described second for the first object described in the Overlapping display in three-dimensional scenic Second object point cloud data of object；

Frame creation module is selected, for selecting frame for the first object-point cloud data creation first and being the second object-point cloud number Frame is selected according to creating second；

Mark module is the first label of the first object point cloud data mark for selecting frame using described first, and utilizes institute It is the second label of the second object point cloud data mark to state second and select frame.

8. device according to claim 7, which is characterized in that it is described to select frame creation module, including：

Acquisition request unit input by user selects frame request to create for obtaining；

Creating unit, for being based on, described to select frame request to create be that the first object-point cloud data creation first selects frame and is institute It states the second object-point cloud data creation second and selects frame.

9. according to any devices of claim 7-8, which is characterized in that the display module, including：

Three-dimensional scenic establishes unit, for based on the first object point cloud data and the second object point cloud data structure three Scene is tieed up, and establishes three-dimensional system of coordinate corresponding with the three-dimensional scenic；

Coordinate transformation unit is used for each laser point in the first object point cloud data and the second object point cloud data Coordinate be converted to the three-dimensional coordinate in the three-dimensional system of coordinate；

Display unit is shown for each laser point to be put into the three-dimensional scenic according to the three-dimensional coordinate of each laser point Show.

10. a kind of readable storage medium storing program for executing, is stored thereon with computer program, which is characterized in that the computer program is handled Device realizes method as claimed in any one of claims 1 to 6 when executing.