CN109816726B - Visual odometer map updating method and system based on depth filter - Google Patents
Visual odometer map updating method and system based on depth filter Download PDFInfo
- Publication number
- CN109816726B CN109816726B CN201910083820.4A CN201910083820A CN109816726B CN 109816726 B CN109816726 B CN 109816726B CN 201910083820 A CN201910083820 A CN 201910083820A CN 109816726 B CN109816726 B CN 109816726B
- Authority
- CN
- China
- Prior art keywords
- frame image
- depth filter
- map
- updating
- seed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000000007 visual effect Effects 0.000 title claims abstract description 25
- 238000009826 distribution Methods 0.000 claims abstract description 29
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 235000013550 pizza Nutrition 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a visual odometer map updating method, a system, a computer readable storage medium and computer equipment based on a depth filter, wherein the updating method comprises the following steps: judging whether a key frame image needs to be added or not according to the acquired depth information characteristic value of the first characteristic point of the current frame image, and if not, updating the seed of the existing depth filter by using the existing depth filter according to the first characteristic point of the current frame image; if the image needs to be added, establishing and initializing a new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter; calculating and updating a probability distribution of the seed, and adding the map to update the map when the depth estimation of the seed converges. The embodiment provided by the invention can obviously accelerate the convergence speed of the depth filter and effectively improve the robustness of the updating of the visual odometer map.
Description
Technical Field
The invention relates to the technical field of visual odometry, in particular to a visual odometry map updating method and system based on a depth filter, a computer readable storage medium and computer equipment.
Background
At present, SLAM (Simultaneous localization and mapping ) is used as a leading-edge technology for establishing an environment model and estimating self-movement, can accurately locate position and posture and construct a map under the condition of no environment prior, and is an important component in the fields of VR, AR, unmanned driving and the like.
Current VR and AR products generally need to be able to evaluate their own motion and position to match the scene content they render, providing the user with a good viewing experience and interaction perception. Therefore, accurate motion estimation and scene model calculation for related products with sensors are very important for the application effect of the products. The Visual odometer (Visual odometer) is one of core modules of the SLAM, and can meet the requirements of current VR and AR products on positioning. However, the current visual odometer method has a slow convergence rate during depth estimation, and often cannot obtain accurate depth estimation after a large amount of information is obtained, so that the final positioning result has weak anti-interference capability and low robustness.
Disclosure of Invention
In order to solve at least one of the above problems, a first aspect of the present invention provides a depth filter-based visual odometry map updating method, including:
judging whether a key frame image needs to be added according to the acquired depth information characteristic value of the first characteristic point of the current frame image,
if the current frame image does not need to be added, updating the seeds of the existing depth filter by using the existing depth filter according to the first feature point of the current frame image;
if the new depth filter needs to be added, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter;
calculating and updating a probability distribution of the seed, and adding the map to update the map when the depth estimation of the seed converges.
Further, the method also comprises
Acquiring and storing a frame image in real time;
extracting a plurality of first characteristic points of the current frame image, calculating the depth information of each first characteristic point and determining the depth information characteristic value of the depth information of all the first characteristic points.
Further, the determining whether to add a key frame image according to the obtained depth information feature value of the first feature point of the current frame image further includes:
and comparing the depth information characteristic value with a preset characteristic value threshold, and if the depth information characteristic value is greater than the characteristic value threshold, adding a key frame image.
Further, if the addition is needed, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image, and updating the seed of the new depth filter according to the key frame image by using the new depth filter further includes:
establishing and initializing a new depth filter, and extracting a second feature point from the current frame image as a seed of the new depth filter;
selecting a preset number of frame images from the stored frame images according to time to be used as key frame images;
and updating the depth information of the seeds in the new depth filter by using the new depth filter according to the preset number of key frame images.
Further, the selecting a preset number of frame images from the stored frame images according to time as key frame images further includes:
selecting key points from the second characteristic points of the current frame image, selecting a plurality of frame images according to time, respectively judging whether each frame image and the current frame image have an overlapping area through the key points, if so, determining the frame image as the key frame image, otherwise, discarding the frame image.
Further, if the adding is not needed, updating the seed of the existing depth filter by using the existing depth filter according to the first feature point of the current frame image further includes:
performing feature matching on the first feature point of the current frame image and the corresponding map point in the map to obtain depth information of the map point;
and the existing depth filter updates the corresponding seeds in the existing depth filter according to the first characteristic point of the current frame image.
Further, the calculating and updating the probability distribution of the seed, the adding the map to update the map when the depth estimation of the seed converges further comprises:
updating the probability distribution of the seeds by using a Bayesian probability model;
and when the probability of the seed is greater than a preset probability threshold value, determining that the seed depth estimation is converged, converting the feature points of the corresponding image frame into map points, and adding the map points to the map to update the map.
The invention provides a visual odometry map updating system based on a depth filter, which comprises:
the key frame selecting device is used for judging whether the system needs to add key frame images or not;
the key frame depth updating device is used for establishing and initializing a new depth filter and determining a seed, and the new depth filter updates the seed of the new depth filter according to the key frame image with the overlapped view field stored by the system;
the depth filter updating device is used for updating the seeds of the existing depth filter of the system according to the current frame image;
and the probability distribution updating device is used for calculating and updating the probability distribution of the seeds, and adding the map to update the map when the depth estimation of the seeds converges.
A third aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method according to the first aspect.
A fourth aspect of the invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method according to the first aspect when executing the program.
The invention has the following beneficial effects:
aiming at the problems of slow convergence time and slow map updating of the existing depth filter, the invention sets a visual odometer map updating method and system based on the depth filter, extracts a second characteristic point and performs depth updating while adding a key frame, thereby fully utilizing the current frame and an overlapped frame with an overlapped view field with the current frame, obviously accelerating the convergence speed of the depth filter and improving the efficiency of depth updating, thereby making up the problems in the prior art and effectively improving the robustness of the visual odometer map updating.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 illustrates a flow diagram of a map update method according to an embodiment of the invention;
FIG. 2 is a flow diagram illustrating the need to add keyframes according to an embodiment of the present invention;
FIG. 3 shows a schematic diagram of the keypoints of one embodiment of the invention;
FIG. 4 is a flow diagram illustrating the elimination of the need to add keyframes according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of a map update system according to an embodiment of the present invention;
fig. 6 shows a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the invention, the invention is further described below with reference to preferred embodiments and the accompanying drawings. Similar parts in the figures are denoted by the same reference numerals. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.
In the prior art, in the process of constructing a map in real time, a depth filter is generally adopted to process an image acquired by a camera through a depth updating method and update the map, specifically, the existing dichotomy method is adopted for updating, new feature points are extracted as seeds when key frames are added, and the operation of depth updating is performed when the next common frame arrives.
Based on the above situation, an embodiment of the present invention provides a depth filter-based visual odometry map updating method, including: judging whether a key frame image needs to be added or not according to the acquired depth information characteristic value of the first characteristic point of the current frame image, and if not, updating the seed of the existing depth filter by using the existing depth filter according to the first characteristic point of the current frame image; if the new depth filter needs to be added, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter; calculating and updating a probability distribution of the seed, and adding the map to update the map when the depth estimation of the seed converges.
In a specific example, as shown in fig. 1, a monocular camera is used to acquire images and construct a map in real time during an actual movement process, the monocular camera acquires and stores frame images according to a preset time interval, for example, 30 frame images are acquired in 1 second, and the images are processed and stored according to a time sequence. The current frame image with the closest time in the collected image is described as follows:
firstly, extracting first feature points of the current frame image, respectively calculating depth information of each feature point, and determining depth information feature values of the depth information of all the first feature points. That is, a depth information feature value is determined from the depth information of the plurality of feature points, where the feature value may be a statistical feature value such as an extreme value, a mean value, and a median value of the depth information, and the feature value is used as a criterion. It should be noted that there are various ways for extracting the feature points, for example, detecting the gray scale difference of 16 pixels around one pixel to extract the feature points, and the application is not limited thereto, and those skilled in the art can extract the feature points according to the requirements of actual resolution and sensitivity.
Secondly, judging whether key frame images need to be added according to the depth information characteristic value. The method specifically comprises the following steps: and comparing the depth information characteristic value with a preset characteristic value threshold, and if the depth information characteristic value is greater than the characteristic value threshold, adding a key frame image. In this embodiment, the median of the depth information is compared with a preset median threshold, and if the median of the depth information is greater than the median threshold, it is determined that a keyframe image needs to be added. The median threshold is a threshold determined according to the acquisition, processing and analysis of an actual frame image, for example, according to the distance between a first feature point of a current frame and a feature point corresponding to the first feature point of an image frame corresponding to a latest map point in the map, when the median of the distances exceeds a preset distance, it is considered that the camera has moved farther, a frame image with an overlapping area needs to be selected from frame images stored before the current frame as a key frame image to be added, and the median threshold can be determined according to the median of the distances under the condition of only considering distance features; for example, the ratio of the coordinate of the stored frame image in the camera coordinate system to the depth median can be calculated on the basis of comparing the distance, if the ratio meets the preset ratio range, the key frame image is considered not to be added, otherwise, the key frame image is added, and therefore the sensitivity of updating the visual odometer map is further improved.
If a key frame image needs to be added, establishing and initializing a new depth filter, extracting a second characteristic point from the current frame image as a seed of the new depth filter, determining the key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter. As shown in fig. 2, the method specifically includes:
and establishing and initializing a new depth filter, and extracting a second feature point from the current frame image as a seed of the new depth filter. Reestablishing a new depth filter, initializing the depth filter, and setting parameters for calculating probability distribution in the depth filter besides conventional initialization; and then, extracting second feature points of the current frame image again, wherein the second feature points are different from the first feature points, the second feature points are used as seeds of a new depth filter, namely each second feature point corresponds to one seed of the new depth filter, and each seed stores the corresponding feature points of which the depth information is not obtained yet for subsequent depth estimation. It should be noted that the present application does not limit the setting of the parameters, so that the calculation of the probability distribution can be realized as a design criterion. The specific parameters set in this example are referred to the Bayesian probabilistic model set forth in "Vogiatzis G, Hernandez C.
And selecting a preset number of frame images from the stored frame images according to time to be used as key frame images. Since the distance between the current frame and the image frame corresponding to the latest map point in the map is long, that is, the camera has moved far, it is necessary to select a key frame image from the frame images stored before the current frame. Specifically, selecting key points from the feature points of the current frame image, judging whether an overlapping area exists between the frame image and the current frame image through the key points, if so, determining that the frame image is a key frame image, and otherwise, discarding the frame image. The preset number may be one frame image or a plurality of frame images, which is not limited in this application.
As shown in fig. 3, in this embodiment, 5 points are selected from the feature points extracted from the current frame as key points, where the 5 key points are composed of 1 central point and 4 boundary points, the central point is a point closest to the center of the image among all the feature points, and the 4 boundary points are points closest to four corners of the image among all the feature points. According to the 5 key points, the frame image with the overlapped area with the current frame image can be selected from the stored frame images, and the frame image is determined as the key frame image. Because the distance between the key frame image and the current frame image is very close, the seeds in the new depth filter are observed by the selected key frame image with high probability, and the key frame images are used for updating the seeds, so that the current frame image is fully utilized, the convergence speed of the depth estimation of the seeds in the new depth filter is accelerated, and the robustness of the updating of the visual odometry map is improved. It should be noted that the selection of the key points is not limited in the present application, and those skilled in the art should select the key points according to actual needs.
And updating the depth information of the seeds in the new depth filter by using the new depth filter according to the preset number of key frame images. Namely, the new depth filter directly updates the stored seeds to obtain the depth information of the seeds, and does not need to wait for a common frame image to come to update the depth information of the seeds according to the common frame image, so that the time consumed by updating the seeds is effectively reduced.
And if the key frame image does not need to be added, updating the seeds of the existing depth filter by using the existing depth filter according to the first characteristic point of the current frame image. And when the key frame image does not need to be added, the current frame image is used as a common frame image, and the seeds stored in the existing depth filter are updated. As shown in fig. 4, the method specifically includes:
firstly, performing feature matching on a first feature point of the current frame and a map point corresponding to the map to obtain depth information of the map point. The method specifically comprises the following steps: the existing depth filter updates the depth information of the seeds stored in the existing depth filter according to the first characteristic point of the current frame image, namely, the map point corresponding to each seed is projected onto the current frame, and the seeds are skipped if the projected point is behind the camera or is not projected onto the current frame, because the current frame cannot influence the updating of the depth information of the seeds. And calculating an affine transformation matrix between the image frame and the current frame corresponding to each seed, namely the affine transformation matrix stored between the source frame and the current frame of each seed in the depth filter. And calculating a determinant of the affine transformation matrix, and searching a pyramid layer with the best matching level in the image pyramid through the value of the determinant.
Secondly, the existing depth filter updates the corresponding seeds in the existing depth filter according to the first feature point of the current frame image. The existing depth filter uses a current frame to evaluate the stored depth information of the seeds, transforms an image block of a current frame image into a reference frame image by using bilinear interpolation, the affine transformation matrix and a pyramid layer to obtain a transformed image block so as to obtain polar lines, samples at intervals on the polar lines, and calculates a difference value and a Jacobian matrix between the image block corresponding to a sampling result and the transformed image block of the current frame, so that a matched characteristic position is accurately predicted, and a more accurate depth estimation value is obtained by using triangulation. Wherein, the reference frame image is a previous common frame image of the current frame image; the image blocks are pixels with preset number around the feature points, and the preset number is set according to actual application requirements. Then, the depth uncertainty of each seed is calculated by using the trigonometric sine and cosine theorem, and the method used in this embodiment is specifically referred to as "pizza M, Forster C, searamuzza d.
Finally, the probability distribution of the seeds is calculated and updated, and the map is added to update the map when the depth estimation of the seeds converges. Whether a new depth filter is established when a key frame image needs to be added, seeds are added in the new depth filter and the key frame image is used for updating the seeds, or the current frame is used as a common frame for updating the seeds stored in the existing depth filter, the probability distribution of each seed is updated by adopting a Bayesian probability model after the seeds are updated. The method specifically comprises the following steps: updating the probability distribution of each seed by using a Bayesian probability model; and when the probability of each seed is greater than a preset probability threshold value, determining that the seed depth estimation is converged, converting the feature point of the corresponding image frame into a map point, and adding the map point to the map to update the map. Namely, when the seed depth estimation converges, the corresponding world coordinate is added to the map as a map point to complete the update of the map. The bayesian probability model used in this embodiment is specifically referred to as "Vogiatzis G, Hernandez c, video-based, real-time multi-view stereo [ J ]", and is specifically represented by jointly representing a series of measured histogram distributions of the depth of each seed by using gaussian distribution and uniform distribution, and by introducing latent variables, the solution of the depth of the seed can be converted into the solution of gaussian distribution and Beta distribution, that is, by using an iterative formula, after adding a new observed quantity, the posterior probability distribution of the seed is updated, and when the seed probability in the posterior probability distribution is greater than a preset probability threshold, the seed depth estimation is considered to be converged, and thus, the map point corresponding to the seed depth estimation is added to the map, so as to update the map information.
In the embodiment, the new depth filter is used for directly evaluating the stored seeds to obtain the depth estimation of the seeds, and the pose estimation is carried out according to the common frame image without waiting for the arrival of the common frame image, so that the time consumed by the depth estimation of the seeds is effectively reduced, the convergence speed of the depth estimation of the seeds is increased, the efficiency of depth updating is improved, the problems in the prior art are solved, and the robustness of the updating of the visual odometer map is effectively improved.
Corresponding to the map updating method provided by the foregoing embodiment, an embodiment of the present application further provides a visual odometry map updating system based on a depth filter, and since the map updating system provided by the embodiment of the present application corresponds to the map updating method provided by the foregoing several embodiments, the foregoing embodiment is also applicable to the map updating system provided by the present embodiment, and detailed description is not given in this embodiment.
As shown in fig. 5, an embodiment of the present application also provides a depth filter-based visual odometry map updating system, including: the key frame selecting device is used for judging whether the system needs to add key frame images or not; the key frame depth updating device is used for establishing and initializing a new depth filter and determining a seed, and the new depth filter updates the seed of the new depth filter according to the key frame image with the overlapped view field stored by the system; the depth filter updating device is used for updating the seeds of the existing depth filter of the system according to the current frame image; and the probability distribution updating device is used for calculating and updating the probability distribution of the seeds, and adding the map to update the map when the depth estimation of the seeds converges.
In a specific example, the key frame selecting device judges whether a key frame image needs to be added according to depth information characteristic values of a plurality of characteristic points extracted from a current frame image, namely whether the camera has moved far or not, and whether the key frame image needs to be selected from frame images stored before the current frame image; if the new depth filter needs to be added, a key frame depth updating device is used for establishing and initializing the new depth filter, the feature points extracted from the current frame image are used as seeds of the new depth filter, and the key frame image with the overlapped view field stored by the system is used for updating the seeds; if the key frame does not need to be added, a depth filter updating device is used, and the existing depth filter of the system is used for updating the seeds stored by the existing depth filter according to the feature points of the current frame image; and finally, calculating the probability distribution of the seeds by adopting a probability distribution updating device no matter the seeds are updated by a new depth filter or the seeds are updated by the existing depth filter, and adding the seeds into the map when the probability distribution of the seeds is larger than a probability threshold value and the estimation of the depth of the seeds is considered to be convergent, thereby realizing the updating of the map.
The system also comprises a characteristic point extracting device and a map point updating device, wherein the characteristic point extracting device is used for extracting a plurality of characteristic points of the current frame image; and when the key frame does not need to be added, before the depth filter updating device updates the stored seeds according to the current frame image by using the existing depth filter, the depth information of the map points of the map is roughly estimated, namely when the current frame image is a common frame image, the feature points extracted from the current frame are subjected to feature matching with the corresponding map points in the map to obtain the depth information of the map points.
The map updating system selects the key frame image from the stored frame images to update the seeds of the depth filter, so that the depth estimation of the seeds is completed by using a small amount of existing information, the pose estimation is performed according to the common frame image without waiting for the arrival of a common frame image, the time consumed by the seed depth estimation is effectively reduced, the convergence rate of the seed depth estimation is accelerated, the depth updating efficiency is improved, the problems in the prior art are solved, the anti-jamming capability is high, the robustness of system operation is remarkably improved, and the map updating system can be widely applied to pose calculation, map construction and scene interaction of VR and AR devices.
Another embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements: judging whether a key frame image needs to be added or not according to the acquired depth information characteristic value of the first characteristic point of the current frame image, and if not, updating the seed of the existing depth filter by using the existing depth filter according to the first characteristic point of the current frame image; if the new depth filter needs to be added, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter; calculating and updating a probability distribution of the seed, and adding the map to update the map when the depth estimation of the seed converges.
In practice, the computer-readable storage medium may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this real-time example, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
As shown in fig. 6, another embodiment of the present invention provides a schematic structural diagram of a computer device. The computer device 12 shown in FIG. 6 is only an example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.
As shown in FIG. 6, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, and commonly referred to as a "hard drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing a depth filter-based visual odometry map update method provided by an embodiment of the present invention.
Aiming at the problems of slow convergence time and slow map updating of the existing depth filter, the invention sets a visual odometer map updating method and system based on the depth filter, extracts a second characteristic point and performs depth updating while adding a key frame, thereby fully utilizing the current frame and an overlapped frame with an overlapped view field with the current frame, obviously accelerating the convergence speed of the depth filter and improving the efficiency of depth updating, thereby making up the problems in the prior art and effectively improving the robustness of the visual odometer map updating.
It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations or modifications may be made on the basis of the above description, and all embodiments may not be exhaustive, and all obvious variations or modifications may be included within the scope of the present invention.
Claims (9)
1. A visual odometry map updating method based on a depth filter is characterized by comprising the following steps:
acquiring and storing a frame image in real time;
judging whether a key frame image needs to be added according to the acquired depth information characteristic value of the first characteristic point of the current frame image,
if the current frame image does not need to be added, updating the seeds of the existing depth filter by using the existing depth filter according to the first feature point of the current frame image;
if the addition is needed, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image by using the new depth filter, wherein the steps of: selecting a preset number of frame images from the stored frame images according to time to serve as key frame images, wherein the preset number of frame images are selected to serve as key frame images, a plurality of frame images are selected according to time, whether an overlapping area exists between each frame image and the current frame image or not is judged through the key points, if yes, the frame images are determined to be the key frame images, and if not, the frame images are discarded;
calculating and updating a probability distribution of the seed, and adding the map to update the map when the depth estimation of the seed converges.
2. The method of claim 1, further comprising
Extracting a plurality of first characteristic points of the current frame image, calculating the depth information of each first characteristic point and determining the depth information characteristic value of the depth information of all the first characteristic points.
3. The method according to claim 1 or 2, wherein the determining whether the key frame image needs to be added according to the acquired depth information feature value of the first feature point of the current frame image further comprises:
and comparing the depth information characteristic value with a preset characteristic value threshold, and if the depth information characteristic value is greater than the characteristic value threshold, adding a key frame image.
4. The method of claim 2, wherein if adding is needed, establishing and initializing a new depth filter, extracting a second feature point from the current frame image as a seed of the new depth filter, determining a key frame image and updating the seed of the new depth filter according to the key frame image using the new depth filter further comprises:
establishing and initializing a new depth filter, and extracting a second feature point from the current frame image as a seed of the new depth filter;
and updating the depth information of the seeds in the new depth filter by using the new depth filter according to the preset number of key frame images.
5. The method of claim 1, wherein if the adding is not required, updating the seed of the existing depth filter according to the first feature point of the current frame image by using the existing depth filter further comprises:
performing feature matching on the first feature point of the current frame image and the corresponding map point in the map to obtain depth information of the map point;
and the existing depth filter updates the corresponding seeds in the existing depth filter according to the first characteristic point of the current frame image.
6. The method of claim 1, wherein the calculating and updating the probability distribution of the seed, wherein adding the map to update the map when the depth estimate of the seed converges further comprises:
updating the probability distribution of the seeds by using a Bayesian probability model;
and when the probability of the seed is greater than a preset probability threshold value, determining that the seed depth estimation is converged, converting the feature points of the corresponding image frame into map points, and adding the map points to the map to update the map.
7. A visual odometry map update system based on a depth filter implementing the visual odometry map update method of any of claims 1-6, comprising:
the key frame selecting device is used for judging whether the system needs to add key frame images or not;
the key frame depth updating device is used for establishing and initializing a new depth filter and determining a seed, and the new depth filter updates the seed of the new depth filter according to the key frame image with the overlapped view field stored by the system;
the depth filter updating device is used for updating the seeds of the existing depth filter of the system according to the current frame image;
and the probability distribution updating device is used for calculating and updating the probability distribution of the seeds, and adding the map to update the map when the depth estimation of the seeds converges.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910083820.4A CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910083820.4A CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109816726A CN109816726A (en) | 2019-05-28 |
| CN109816726B true CN109816726B (en) | 2021-10-01 |
Family
ID=66605525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910083820.4A Active CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109816726B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309330B (en) * | 2019-07-01 | 2021-08-31 | 北京百度网讯科技有限公司 | Processing method and device of visual map |
| CN111288989B (en) * | 2020-02-25 | 2021-11-05 | 浙江大学 | Visual positioning method for small unmanned aerial vehicle |
| CN116197889A (en) * | 2021-11-30 | 2023-06-02 | 珠海一微半导体股份有限公司 | A Positioning Method for Ceiling Vision Robot |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108151728A (en) * | 2017-12-06 | 2018-06-12 | 华南理工大学 | A kind of half dense cognitive map creation method for binocular SLAM |
| CN108615246A (en) * | 2018-04-19 | 2018-10-02 | 浙江大承机器人科技有限公司 | It improves visual odometry system robustness and reduces the method that algorithm calculates consumption |
| CN108986037A (en) * | 2018-05-25 | 2018-12-11 | 重庆大学 | Monocular vision odometer localization method and positioning system based on semi-direct method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107687850B (en) * | 2017-07-26 | 2021-04-23 | 哈尔滨工业大学深圳研究生院 | A Pose and Attitude Estimation Method for Unmanned Aircraft Based on Vision and Inertial Measurement Unit |
-
2019
- 2019-01-29 CN CN201910083820.4A patent/CN109816726B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108151728A (en) * | 2017-12-06 | 2018-06-12 | 华南理工大学 | A kind of half dense cognitive map creation method for binocular SLAM |
| CN108615246A (en) * | 2018-04-19 | 2018-10-02 | 浙江大承机器人科技有限公司 | It improves visual odometry system robustness and reduces the method that algorithm calculates consumption |
| CN108986037A (en) * | 2018-05-25 | 2018-12-11 | 重庆大学 | Monocular vision odometer localization method and positioning system based on semi-direct method |
Non-Patent Citations (1)
| Title |
|---|
| SVO: Fast Semi-Direct Monocular Visual Odometry;Christian Forster et al.;《2014 IEEE International Conference on Robotics & Automation》;20140929;第15-22页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109816726A (en) | 2019-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109345596B (en) | Multi-sensor calibration method, device, computer equipment, medium and vehicle | |
| CN111325796B (en) | Method and apparatus for determining pose of vision equipment | |
| CN110322500B (en) | Optimization method and device, medium and electronic equipment for real-time positioning and map construction | |
| CN109188457B (en) | Object detection frame generation method, device, equipment, storage medium and vehicle | |
| CN110766716B (en) | Method and system for acquiring information of space unknown moving target | |
| CN105809687B (en) | A Monocular Vision Odometry Method Based on Edge Point Information in Image | |
| WO2021081808A1 (en) | Artificial neural network-based object detection system and method | |
| CN113763466B (en) | Loop detection method and device, electronic equipment and storage medium | |
| CN109816726B (en) | Visual odometer map updating method and system based on depth filter | |
| CN111982124B (en) | 3D LiDAR Navigation Method and Device in Glass Scene Based on Deep Learning | |
| CN114612572B (en) | A laser radar and camera extrinsic parameter calibration method and device based on deep learning | |
| CN114004972A (en) | Image semantic segmentation method, device, equipment and storage medium | |
| US12254041B2 (en) | Position recognition method and system based on visual information processing | |
| CN110349212A (en) | Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring | |
| WO2022160897A1 (en) | Binocular parallax estimation method, model training method and related device | |
| CN114627438A (en) | Target detection model generation method, target detection method, device and medium | |
| CN113112542A (en) | Visual positioning method and device, electronic equipment and storage medium | |
| WO2024001083A1 (en) | Localization method, apparatus and device, and storage medium | |
| CN114299115A (en) | Method and device for multi-target tracking, storage medium and electronic device | |
| CN113932796A (en) | High-precision map lane line generation method and device and electronic equipment | |
| CN117315015A (en) | Robot pose determining method and device, medium and electronic equipment | |
| CN109840598B (en) | A method and device for establishing a deep learning network model | |
| CN115775387A (en) | Target detection method, device, equipment and storage medium | |
| CN110688500B (en) | Database construction method, positioning method and related equipment thereof | |
| CN113763468B (en) | Positioning method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |