CN110599542A - Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area - Google Patents
Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area Download PDFInfo
- Publication number
- CN110599542A CN110599542A CN201910817262.XA CN201910817262A CN110599542A CN 110599542 A CN110599542 A CN 110599542A CN 201910817262 A CN201910817262 A CN 201910817262A CN 110599542 A CN110599542 A CN 110599542A
- Authority
- CN
- China
- Prior art keywords
- local
- points
- mapping
- dynamic noise
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 21
- 239000011159 matrix material Substances 0.000 claims abstract description 40
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims abstract description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 30
- 230000004044 response Effects 0.000 claims description 23
- 238000001914 filtration Methods 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 230000006978 adaptation Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 description 11
- 238000004590 computer program Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000000007 visual effect Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 238000010276 construction Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method and a device for local mapping of adaptive VSLAM facing to a geometric region, and belongs to the field of mapping. The method comprises the following steps: acquiring an experimental data set, and eliminating dynamic noise points of images in the experimental data set by adopting RCNN (recursive least squares NN); inputting the image without the dynamic noise points into a CNN (continuous noise network), determining attributes of local point clouds and voxels in a local space, calculating a homography matrix through a pixel matching and back propagation algorithm, and estimating the pose of a camera according to the homography matrix; and carrying out local mapping according to the camera pose, and carrying out self-adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels. The device includes: the device comprises an acquisition module, a calculation module and a mapping module. The method has the advantages of high calculation efficiency, high accuracy and strong generalization capability, and enhances the effectiveness of the characteristics, so that the output result has higher credibility.
Description
Technical Field
The present application relates to the field of map construction, and in particular, to a method and an apparatus for local mapping of a geometric-region-oriented adaptive VSLAM.
Background
The VSLAM (Visual simultaneouslocalization And Mapping) refers to a process of calculating a self position And constructing an environment map according to information of a Visual sensor, And can solve the problems of positioning And map construction during movement in an unknown environment, And is more accurate And rapid. The VSLAM model mainly comprises sensor data preprocessing, a front end, a back end, loop detection and graph building. The front end is also called VO (Visual odometer), and mainly studies how to quantitatively estimate the motion of the inter-frame camera according to the adjacent frame images. The motion trail of the camera carrier (such as a robot or a vehicle) is formed by connecting the motion trails of the adjacent frames, and the positioning problem is solved. And then, according to the estimated position of the camera at each moment, calculating the position of a space point of each pixel, and completing the construction of the map.
The output result of the VSLAM model is mainly influenced by conditions such as real-time performance, environment, illumination and the like, and the visual odometer is easy to generate accumulated errors, so that the accuracy is reduced, and further, the accuracy of the constructed graph is also influenced to a certain degree.
Disclosure of Invention
It is an object of the present application to overcome the above problems or to at least partially solve or mitigate the above problems.
According to an aspect of the present application, there is provided a method for local mapping of adaptive VSLAM for a geometric region, including:
acquiring an experimental data set, and eliminating dynamic noise points of images in the experimental data set by adopting an RCNN (convolutional neural network) region convolutional neural network;
inputting the image without the dynamic noise points into a CNN convolutional neural network, determining attributes of local point clouds and voxels in a local space, calculating a homography matrix through a pixel matching and back propagation algorithm, and estimating the pose of the camera according to the homography matrix;
and carrying out local mapping according to the camera pose, and carrying out self-adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels.
Optionally, the homography matrix is calculated by a pixel matching and back propagation algorithm, including:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
Optionally, performing adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels, including:
dividing the local point cloud into a plurality of sub-areas, acquiring response intensity from the attribute of the voxel, and filtering the feature points in each sub-area according to the response intensity to obtain a local feature map;
and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, expanding the local feature map by increasing edge adjacent voxel space and/or angle adjacent voxel space for the sub-area where the registration points in the local point cloud are located.
Optionally, filtering the feature points in each sub-region according to the response strength to obtain a local feature map, including:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
Optionally, for the images in the experimental data set, removing dynamic noise points by using an RCNN area convolution neural network, including:
by means of RCNN combined with geometric features, for each current frame, finding N key frames with the highest overlapping degree, acquiring the projection of feature points of the key frames on the current frame, and calculating the position and depth value of the projection; and judging whether the depth value exceeds a set threshold value, if so, judging that the feature point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and removing the dynamic noise point of which the variance is smaller than a specified value.
According to another aspect of the present application, there is provided a local mapping apparatus for adaptive VSLAM facing a geometric region, including:
the acquisition module is configured to acquire an experimental data set, and eliminate dynamic noise points of images in the experimental data set by adopting an RCNN (convolutional neural network) region convolution neural network;
the calculation module is configured to input the image with the dynamic noise points removed into a CNN convolutional neural network, determine attributes of local point clouds and voxels in local space, calculate a homography matrix through a pixel matching and back propagation algorithm, and estimate a camera pose according to the homography matrix;
a mapping module configured to perform local mapping according to the camera pose, the local mapping being adaptively optimized according to the attributes of the local point cloud and the voxels.
Optionally, the computing module is specifically configured to:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
Optionally, the mapping module includes:
a mapping unit configured to perform local mapping according to the camera pose;
the self-adaptive unit is configured to divide the local point cloud into a plurality of sub-areas, obtain response intensity from attributes of the voxels, and filter feature points in each sub-area according to the response intensity to obtain a local feature map; and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, expanding the local feature map by increasing edge adjacent voxel space and/or angle adjacent voxel space for the sub-area where the registration points in the local point cloud are located.
Optionally, the adaptation unit is specifically configured to:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
Optionally, the obtaining module is specifically configured to:
by means of RCNN combined with geometric features, for each current frame, finding N key frames with the highest overlapping degree, acquiring the projection of feature points of the key frames on the current frame, and calculating the position and depth value of the projection; and judging whether the depth value exceeds a set threshold value, if so, judging that the feature point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and removing the dynamic noise point of which the variance is smaller than a specified value.
According to yet another aspect of the application, there is provided a computing device comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.
According to yet another aspect of the application, a computer-readable storage medium, preferably a non-volatile readable storage medium, is provided, having stored therein a computer program which, when executed by a processor, implements a method as described above.
According to yet another aspect of the application, there is provided a computer program product comprising computer readable code which, when executed by a computer device, causes the computer device to perform the method described above.
According to the technical scheme, the experimental data set is obtained, dynamic noise points of an image in the experimental data set are eliminated through RCNN, the image is input into CNN, attributes of local point cloud and voxels in a local space are determined, a homography matrix is calculated through pixel matching and a back propagation algorithm, the camera pose is estimated according to the homography matrix, local mapping is conducted according to the camera pose, self-adaptive optimization is conducted on the local mapping according to the attributes of the local point cloud and the voxels, the calculation is efficient, the accuracy rate is high, the generalization capability is strong, the feature effectiveness is enhanced, and the output result is enabled to be more credible. In addition, the accuracy of the output result of the model can be further improved by rejecting dynamic noise data.
The above and other objects, advantages and features of the present application will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the present application will be described in detail hereinafter by way of illustration and not limitation with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
FIG. 1 is a flow diagram of a geometric region-oriented adaptive VSLAM local mapping method according to one embodiment of the present application;
FIG. 2 is a flow diagram of a geometric region-oriented adaptive VSLAM local mapping method according to another embodiment of the present application;
FIG. 3 is a flow diagram illustrating a geometric region-oriented adaptive VSLAM local building block according to another embodiment of the present application;
fig. 4 is a block diagram of an adaptive VSLAM local mapping apparatus for geometric area according to another embodiment of the present application;
FIG. 5 is a block diagram of a computing device according to another embodiment of the present application;
fig. 6 is a diagram of a computer-readable storage medium structure according to another embodiment of the present application.
Detailed Description
Fig. 1 is a flow chart of a geometric region-oriented adaptive VSLAM local mapping method according to one embodiment of the present application. Referring to fig. 1, the method includes:
101: acquiring an experimental data set, and removing dynamic noise points of images in the experimental data set by adopting RCNN (Regions with a connected Neural Network Features);
102: inputting the image without the dynamic noise points into a CNN (Convolutional Neural Networks), determining attributes of local point clouds and voxels in a local space, calculating a homography matrix through a pixel matching and back propagation algorithm, and estimating the pose of the camera according to the homography matrix;
103: and performing local mapping according to the pose of the camera, and performing self-adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels.
In the method provided by this embodiment, a VO visual odometry process is performed, two adjacent key frame images are input in the process, a plurality of VO processes are performed in practical application, each VO process processes two adjacent key frame images, and a detailed process is the same as the above process and is not described herein again.
In this embodiment, optionally, the calculating the homography matrix through the pixel matching and back propagation algorithm includes:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
In this embodiment, optionally, the performing adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels includes:
dividing the local point cloud into a plurality of sub-areas, acquiring response intensity from the attribute of the voxel, and filtering the feature points in each sub-area according to the response intensity to obtain a local feature map;
and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, increasing edge adjacent voxel space and/or angle adjacent voxel space of the sub-region where the registration points in the local point cloud are located, and expanding the local feature map.
In this embodiment, optionally, the filtering the feature points in each sub-region according to the response strength to obtain a local feature map includes:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
In this embodiment, optionally, removing dynamic noise points from an image in the experimental data set by using an RCNN area convolution neural network includes:
by means of RCNN combined with geometric features, for each current frame, N key frames with the highest overlapping degree are found, projection of feature points of the key frames on the current frame is obtained, and the position and the depth value of the projection are calculated; judging whether the depth value exceeds a set threshold value, if so, judging that the characteristic point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and eliminating the dynamic noise point of which the variance is smaller than a specified value.
According to the method provided by the embodiment, the experimental data set is obtained, dynamic noise points of the image are removed through RCNN, the image is input into CNN, attributes of local point cloud and voxels in a local space are determined, the homography matrix is calculated through pixel matching and a back propagation algorithm, the camera pose is estimated according to the homography matrix, local mapping is carried out according to the camera pose, self-adaptive optimization is carried out on the local mapping according to the attributes of the local point cloud and the voxels, the calculation is efficient, the accuracy rate is high, the generalization capability is strong, the feature effectiveness is enhanced, and the output result is more credible. In addition, the accuracy of the output result of the model can be further improved by rejecting dynamic noise data.
Fig. 2 is a flow diagram of a geometric region-oriented adaptive VSLAM local mapping method according to another embodiment of the present application. Referring to fig. 2, the method includes:
201: acquiring an experimental data set, and eliminating dynamic noise points of images in the experimental data set by adopting RCNN (recursive least squares NN);
in this embodiment, preferably, the selected experimental data set is a KITTI data set (jointly created by the charles stuuer institute of technology, germany and the technical research institute of yota america), and is a computer vision algorithm evaluation data set in the current international largest automatic driving scene. The acquisition platform of KITTI data set includes: 2 grayscale cameras, 2 color cameras, one Velodyne 3D lidar, 4 optical lenses, and 1 GPS navigation system. The entire data set consisted of 389 images of stereoscopic images and optical flow maps, 39.2 km visual ranging sequence and over 200,0003D labeled objects, where each image included a maximum of 15 vehicles and 30 pedestrians, and also contained varying degrees of occlusion.
In this embodiment, optionally, the removing dynamic noise points from the images in the experimental data set by using the RCNN includes:
by means of RCNN combined with geometric features, for each current frame, N key frames with the highest overlapping degree are found, projection of feature points of the key frames on the current frame is obtained, and the position and the depth value of the projection are calculated; judging whether the depth value exceeds a set threshold value, if so, judging that the characteristic point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and eliminating the dynamic noise point of which the variance is smaller than a specified value.
Wherein, N may be set to 5 or other values, and is not limited specifically.
202: inputting the image without the dynamic noise points into a CNN (continuous noise network), and determining attributes of local point clouds and voxels in a local space;
203: setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and calculating the homography matrix by executing a back propagation algorithm according to the loss function;
204: estimating the camera pose according to the homography matrix, and performing local image building according to the camera pose;
205: dividing the local point cloud into a plurality of sub-areas, acquiring response intensity from attributes of voxels, reserving feature points with response intensity greater than a specified value in each sub-area as a feature map, and filtering out other feature points;
206: and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, increasing edge adjacent voxel space and/or angle adjacent voxel space of the sub-region where the registration points in the local point cloud are located, and expanding the local feature map.
Fig. 3 is a flow diagram illustrating a geometric region-oriented adaptive VSLAM local building process according to another embodiment of the present application. Referring to fig. 3, an image in an experimental data set is input, dynamic noise points are removed by using the RCNN, then the CNN is input, attributes of local point clouds and voxels in a local space are determined, homography matrixes are calculated by using a pixel matching and back propagation algorithm, then the camera pose is estimated and local mapping is performed, then self-adaptive optimization is performed on the local mapping according to the attributes of the local point clouds and the voxels, an output result is obtained, the calculation efficiency is high, the accuracy rate is high, the generalization capability is strong, the feature effectiveness is enhanced, and the output result has higher credibility.
According to the method provided by the embodiment, the experimental data set is obtained, dynamic noise points of the image are removed through RCNN, the image is input into CNN, attributes of local point cloud and voxels in a local space are determined, the homography matrix is calculated through pixel matching and a back propagation algorithm, the camera pose is estimated according to the homography matrix, local mapping is carried out according to the camera pose, self-adaptive optimization is carried out on the local mapping according to the attributes of the local point cloud and the voxels, the calculation is efficient, the accuracy rate is high, the generalization capability is strong, the feature effectiveness is enhanced, and the output result is more credible. In addition, the accuracy of the output result of the model can be further improved by rejecting dynamic noise data.
Fig. 4 is a block diagram of a geometry-area-oriented adaptive VSLAM local mapping apparatus according to another embodiment of the present application. Referring to fig. 4, the apparatus includes:
an obtaining module 401 configured to obtain an experimental data set, and for an image in the experimental data set, remove a dynamic noise point by using an RCNN region convolution neural network;
a calculation module 402 configured to input the image from which the dynamic noise points are removed into a CNN convolutional neural network, determine attributes of local point clouds and voxels in a local space, calculate a homography matrix through a pixel matching and back propagation algorithm, and estimate a camera pose according to the homography matrix;
and a mapping module 403 configured to perform local mapping according to the camera pose, and perform adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels.
In this embodiment, optionally, the computing module is specifically configured to:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
In this embodiment, optionally, the mapping module includes:
a mapping unit configured to perform local mapping according to a camera pose;
the adaptive unit is configured to divide the local point cloud into a plurality of sub-areas, acquire response intensity from attributes of voxels, and filter feature points in each sub-area according to the response intensity to obtain a local feature map; and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, increasing edge adjacent voxel space and/or angle adjacent voxel space of the sub-region where the registration points in the local point cloud are located, and expanding the local feature map.
In this embodiment, optionally, the adaptive unit is specifically configured to:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
In this embodiment, optionally, the obtaining module is specifically configured to:
by means of RCNN combined with geometric features, for each current frame, N key frames with the highest overlapping degree are found, projection of feature points of the key frames on the current frame is obtained, and the position and the depth value of the projection are calculated; judging whether the depth value exceeds a set threshold value, if so, judging that the characteristic point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and eliminating the dynamic noise point of which the variance is smaller than a specified value.
The apparatus provided in this embodiment may perform the method provided in any of the above method embodiments, and details of the process are described in the method embodiments and are not described herein again.
According to the device provided by the embodiment, the experimental data set is obtained, dynamic noise points of an image in the experimental data set are removed through RCNN, the image is input into CNN, attributes of local point cloud and voxels in a local space are determined, a homography matrix is calculated through pixel matching and a back propagation algorithm, the camera pose is estimated according to the homography matrix, local mapping is carried out according to the camera pose, self-adaptive optimization is carried out on the local mapping according to the attributes of the local point cloud and the voxels, the calculation is efficient, the accuracy rate is high, the generalization capability is strong, the feature effectiveness is enhanced, and the output result is enabled to be more credible. In addition, the accuracy of the output result of the model can be further improved by rejecting dynamic noise data.
The above and other objects, advantages and features of the present application will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Embodiments also provide a computing device, referring to fig. 5, comprising a memory 1120, a processor 1110 and a computer program stored in said memory 1120 and executable by said processor 1110, the computer program being stored in a space 1130 for program code in the memory 1120, the computer program, when executed by the processor 1110, implementing the method steps 1131 for performing any of the methods according to the invention.
The embodiment of the application also provides a computer readable storage medium. Referring to fig. 6, the computer readable storage medium comprises a storage unit for program code provided with a program 1131' for performing the steps of the method according to the invention, which program is executed by a processor.
The embodiment of the application also provides a computer program product containing instructions. Which, when run on a computer, causes the computer to carry out the steps of the method according to the invention.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, cause the computer to perform, in whole or in part, the procedures or functions described in accordance with the embodiments of the application. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by a program, and the program may be stored in a computer-readable storage medium, where the storage medium is a non-transitory medium, such as a random access memory, a read only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape (magnetic tape), a floppy disk (floppy disk), an optical disk (optical disk), and any combination thereof.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A local mapping method for adaptive VSLAM facing to a geometric region comprises the following steps:
acquiring an experimental data set, and eliminating dynamic noise points of images in the experimental data set by adopting an RCNN (convolutional neural network) region convolutional neural network;
inputting the image without the dynamic noise points into a CNN convolutional neural network, determining attributes of local point clouds and voxels in a local space, calculating a homography matrix through a pixel matching and back propagation algorithm, and estimating the pose of the camera according to the homography matrix;
and carrying out local mapping according to the camera pose, and carrying out self-adaptive optimization on the local mapping according to the attributes of the local point cloud and the voxels.
2. The method of claim 1, wherein computing the homography matrix by a pixel matching and back propagation algorithm comprises:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
3. The method of claim 1, wherein adaptively optimizing the local map according to attributes of the local point cloud and the voxels comprises:
dividing the local point cloud into a plurality of sub-areas, acquiring response intensity from the attribute of the voxel, and filtering the feature points in each sub-area according to the response intensity to obtain a local feature map;
and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, expanding the local feature map by increasing edge adjacent voxel space and/or angle adjacent voxel space for the sub-area where the registration points in the local point cloud are located.
4. The method of claim 3, wherein filtering the feature points in each sub-region according to the response strength to obtain a local feature map comprises:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
5. The method according to any one of claims 1 to 4, wherein the step of removing dynamic noise points from the images in the experimental data set by using an RCNN (convolutional neural network) comprises the following steps:
by means of RCNN combined with geometric features, for each current frame, finding N key frames with the highest overlapping degree, acquiring the projection of feature points of the key frames on the current frame, and calculating the position and depth value of the projection; and judging whether the depth value exceeds a set threshold value, if so, judging that the feature point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and removing the dynamic noise point of which the variance is smaller than a specified value.
6. A local mapping device of adaptive VSLAM facing to a geometric region comprises:
the acquisition module is configured to acquire an experimental data set, and eliminate dynamic noise points of images in the experimental data set by adopting an RCNN (convolutional neural network) region convolution neural network;
the calculation module is configured to input the image with the dynamic noise points removed into a CNN convolutional neural network, determine attributes of local point clouds and voxels in local space, calculate a homography matrix through a pixel matching and back propagation algorithm, and estimate a camera pose according to the homography matrix;
a mapping module configured to perform local mapping according to the camera pose, the local mapping being adaptively optimized according to the attributes of the local point cloud and the voxels.
7. The apparatus of claim 6, wherein the computing module is specifically configured to:
and setting the loss function as the distance between the feature point of the previous frame and the corresponding feature point of the current frame after the change of the homography matrix, and executing a back propagation algorithm according to the loss function to calculate the homography matrix.
8. The apparatus of claim 6, wherein the mapping module comprises:
a mapping unit configured to perform local mapping according to the camera pose;
the self-adaptive unit is configured to divide the local point cloud into a plurality of sub-areas, obtain response intensity from attributes of the voxels, and filter feature points in each sub-area according to the response intensity to obtain a local feature map; and judging whether the number of the feature points in the local point cloud is smaller than the preset number q of the registration points, if so, expanding the local feature map by increasing edge adjacent voxel space and/or angle adjacent voxel space for the sub-area where the registration points in the local point cloud are located.
9. The apparatus of claim 8, wherein the adaptation unit is specifically configured to:
and in each sub-area, reserving the characteristic points with the response intensity larger than the specified value as a characteristic map, and filtering out the rest characteristic points.
10. The apparatus according to any of claims 6-9, wherein the acquisition module is specifically configured to:
by means of RCNN combined with geometric features, for each current frame, finding N key frames with the highest overlapping degree, acquiring the projection of feature points of the key frames on the current frame, and calculating the position and depth value of the projection; and judging whether the depth value exceeds a set threshold value, if so, judging that the feature point is a dynamic noise point, calculating the variance between the dynamic noise point and the surrounding points, and removing the dynamic noise point of which the variance is smaller than a specified value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817262.XA CN110599542A (en) | 2019-08-30 | 2019-08-30 | Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817262.XA CN110599542A (en) | 2019-08-30 | 2019-08-30 | Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110599542A true CN110599542A (en) | 2019-12-20 |
Family
ID=68857043
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910817262.XA Pending CN110599542A (en) | 2019-08-30 | 2019-08-30 | Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110599542A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113503883A (en) * | 2021-06-22 | 2021-10-15 | 北京三快在线科技有限公司 | Method for collecting data for constructing map, storage medium and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103268729A (en) * | 2013-05-22 | 2013-08-28 | 北京工业大学 | A method for creating cascaded maps for mobile robots based on hybrid features |
| CN106658023A (en) * | 2016-12-21 | 2017-05-10 | 山东大学 | End-to-end visual odometer and method based on deep learning |
| CN107945265A (en) * | 2017-11-29 | 2018-04-20 | 华中科技大学 | Real-time dense monocular SLAM method and systems based on on-line study depth prediction network |
| CN109341694A (en) * | 2018-11-12 | 2019-02-15 | 哈尔滨理工大学 | An autonomous positioning and navigation method for a mobile detection robot |
| CN109387204A (en) * | 2018-09-26 | 2019-02-26 | 东北大学 | The synchronous positioning of the mobile robot of dynamic environment and patterning process in faced chamber |
| CN109816686A (en) * | 2019-01-15 | 2019-05-28 | 山东大学 | Robot semantic SLAM method, processor and robot based on object instance matching |
| CN110009683A (en) * | 2019-03-29 | 2019-07-12 | 北京交通大学 | Real-time object detection method on plane based on MaskRCNN |
| CN110135480A (en) * | 2019-04-30 | 2019-08-16 | 南开大学 | A network data learning method based on unsupervised object detection to eliminate bias |
-
2019
- 2019-08-30 CN CN201910817262.XA patent/CN110599542A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103268729A (en) * | 2013-05-22 | 2013-08-28 | 北京工业大学 | A method for creating cascaded maps for mobile robots based on hybrid features |
| CN106658023A (en) * | 2016-12-21 | 2017-05-10 | 山东大学 | End-to-end visual odometer and method based on deep learning |
| CN107945265A (en) * | 2017-11-29 | 2018-04-20 | 华中科技大学 | Real-time dense monocular SLAM method and systems based on on-line study depth prediction network |
| CN109387204A (en) * | 2018-09-26 | 2019-02-26 | 东北大学 | The synchronous positioning of the mobile robot of dynamic environment and patterning process in faced chamber |
| CN109341694A (en) * | 2018-11-12 | 2019-02-15 | 哈尔滨理工大学 | An autonomous positioning and navigation method for a mobile detection robot |
| CN109816686A (en) * | 2019-01-15 | 2019-05-28 | 山东大学 | Robot semantic SLAM method, processor and robot based on object instance matching |
| CN110009683A (en) * | 2019-03-29 | 2019-07-12 | 北京交通大学 | Real-time object detection method on plane based on MaskRCNN |
| CN110135480A (en) * | 2019-04-30 | 2019-08-16 | 南开大学 | A network data learning method based on unsupervised object detection to eliminate bias |
Non-Patent Citations (3)
| Title |
|---|
| BERTA BESCOS等: ""DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes"", 《IEEE ROBOTICS AND AUTOMATION LETTERS》 * |
| SUDHEENDRA VIJAYANARASIMHAN等: ""SfM-Net: Learning of Structure and Motion from Video"", 《ARXIV》 * |
| 张峻宁等: ""一种自适应特征地图匹配的改进VSLAM算法"", 《自动化学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113503883A (en) * | 2021-06-22 | 2021-10-15 | 北京三快在线科技有限公司 | Method for collecting data for constructing map, storage medium and electronic equipment |
| CN113503883B (en) * | 2021-06-22 | 2022-07-19 | 北京三快在线科技有限公司 | Method for collecting data for constructing map, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7106665B2 (en) | MONOCULAR DEPTH ESTIMATION METHOD AND DEVICE, DEVICE AND STORAGE MEDIUM THEREOF | |
| CN111209770B (en) | Lane line recognition method and device | |
| JP6095018B2 (en) | Detection and tracking of moving objects | |
| CN112947419B (en) | Obstacle avoidance method, device and equipment | |
| JP7209115B2 (en) | Detection, 3D reconstruction and tracking of multiple rigid objects moving in relatively close proximity | |
| EP3293700B1 (en) | 3d reconstruction for vehicle | |
| CN111322993B (en) | Visual positioning method and device | |
| CN110262487B (en) | Obstacle detection method, terminal and computer readable storage medium | |
| CN112509003B (en) | Method and system for solving target tracking frame drift | |
| CN110619299A (en) | Object recognition SLAM method and device based on grid | |
| CN110992424B (en) | Positioning method and system based on binocular vision | |
| CN115187941A (en) | Target detection and positioning method, system, device and storage medium | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| CN113450385A (en) | Night work engineering machine vision tracking method and device and storage medium | |
| CN119224743B (en) | Laser radar and camera external parameter calibration method | |
| CN111428651A (en) | Vehicle obstacle information acquisition method and system and vehicle | |
| CN115761668A (en) | Camera stain recognition method and device, vehicle and storage medium | |
| CN108876807B (en) | Real-time satellite-borne satellite image moving object detection tracking method | |
| CN110599542A (en) | Method and device for local mapping of adaptive VSLAM (virtual local area model) facing to geometric area | |
| CN112308917B (en) | A vision-based mobile robot positioning method | |
| CN118311955A (en) | Unmanned aerial vehicle control method, terminal, unmanned aerial vehicle and storage medium | |
| CN115511970B (en) | Visual positioning method for autonomous parking | |
| CN116012421A (en) | Target tracking method and device | |
| CN115752489B (en) | Positioning method and device of movable equipment and electronic equipment | |
| Jaspers et al. | Fast and robust b-spline terrain estimation for off-road navigation with stereo vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191220 |
|
| RJ01 | Rejection of invention patent application after publication |