CN118642121B - Monocular vision ranging and laser point cloud fusion space positioning method and system - Google Patents
Monocular vision ranging and laser point cloud fusion space positioning method and system Download PDFInfo
- Publication number
- CN118642121B CN118642121B CN202411124770.7A CN202411124770A CN118642121B CN 118642121 B CN118642121 B CN 118642121B CN 202411124770 A CN202411124770 A CN 202411124770A CN 118642121 B CN118642121 B CN 118642121B
- Authority
- CN
- China
- Prior art keywords
- image
- dimensional
- positioning
- acquisition
- tag
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000004927 fusion Effects 0.000 title claims abstract description 24
- 238000004458 analytical method Methods 0.000 claims abstract description 43
- 230000000007 visual effect Effects 0.000 claims abstract description 17
- 238000011156 evaluation Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 230000007613 environmental effect Effects 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000004364 calculation method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 3
- 238000011960 computer-aided design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000282806 Rhinoceros Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application provides a monocular vision ranging and laser point cloud fusion space positioning method and system, which relate to the technical field of space positioning and are used for receiving a two-dimensional image of a positioning target through a monocular sensor; loading a positioning target three-dimensional model, and carrying out two-dimensional projection to obtain a two-dimensional projection slice image; performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image; performing geometric positioning to obtain an image acquisition characteristic distance; collecting laser point clouds through a laser scanner to construct a three-dimensional environment model; and positioning the three-dimensional model of the positioning target on the three-dimensional environment model according to the monocular sensor coordinate identification by the image acquisition characteristic distance, and performing visual display through a user side. The application solves the technical problem that the monocular positioning technology lacks depth information and global environment information to cause the limitation of positioning accuracy and environment adaptability, realizes high-precision spatial positioning and enhances the environment adaptability.
Description
Technical Field
The application relates to the technical field of space positioning, in particular to a space positioning method and a system for monocular vision ranging and laser point cloud fusion.
Background
In modern positioning technology, monocular vision ranging is a widely used method to estimate the distance between a target object and a camera by analyzing image information. Such methods typically rely on image processing and computer vision techniques, such as feature-based methods and deep learning-based methods. Feature-based methods estimate distance by detecting and matching feature points in an image, while depth-learning-based methods predict depth information by training a large amount of image data.
Although the monocular vision ranging technique is low in cost and easy to implement, it is susceptible to factors such as illumination variation, occlusion and parallax in complex or sparse texture environments, resulting in limited positioning accuracy. Particularly, under the condition of lacking depth information, the monocular positioning technology is difficult to realize accurate estimation of a target object, and due to the lack of global environment information, a positioning result is often cracked, and effective analysis of the state of a positioning target in a macroscopic environment is difficult. In a structured environment, such as an indoor or obviously characterized outdoor scene, texture and lighting conditions in the environment may cause the accuracy of feature point matching to be reduced, thereby affecting the performance of monocular vision ranging.
Disclosure of Invention
The application provides a monocular vision ranging and laser point cloud fusion space positioning method and system, which solve the technical problem that the monocular positioning technology lacks depth information and global environment information to cause the limitation of positioning accuracy and environmental adaptability, and by combining the monocular vision ranging and laser point cloud technology, the accuracy and reliability of space positioning are improved, and meanwhile, the monocular vision ranging and laser point cloud fusion space positioning system is better fused with the actual environment, and the environmental adaptability is enhanced.
In view of the above problems, in one aspect, the present application provides a spatial positioning method for combining monocular vision ranging with laser point cloud, the method comprising: receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag; loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image; performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image has an actual size label; performing geometric positioning according to the acquisition focal length label, the actual size label and the image size label to obtain an image acquisition characteristic distance; acquiring laser point clouds through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark; and positioning the three-dimensional model of the positioning target at the three-dimensional environment model according to the monocular sensor coordinate mark by the image acquisition characteristic distance, and performing visual display through a user side.
On the other hand, the application also provides a space positioning system for fusion of monocular vision ranging and laser point cloud, which comprises: the monocular ranging module is used for receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag; the two-dimensional projection module is used for loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image; the shape analysis module is used for carrying out shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image is provided with an actual size label; the geometric positioning module is used for performing geometric positioning according to the acquisition focal length label, the actual size label and the image size label to obtain an image acquisition characteristic distance; the three-dimensional environment model construction module is used for acquiring laser point clouds through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark; and the target positioning module is used for positioning the three-dimensional model of the positioning target to the three-dimensional environment model according to the monocular sensor coordinate identification and the image acquisition characteristic distance, and performing visual display through a user side.
One or more technical schemes provided by the application have at least the following technical effects or advantages:
and receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag. These tags provide the shooting location of the image, the focal length of the camera, and the size information of the image, providing the basis data for subsequent positioning calculations. Loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image; this step converts the three-dimensional information into a two-dimensional image, generating a slice image that coincides with the actual scene for comparison with the image acquired by the monocular sensor. And carrying out shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image has an actual size label. This step ensures that the selected two-dimensional projection slice image is exactly matched with the actual image by shape analysis, providing reliable dimensional information and providing an accurate data basis for geometric positioning. And performing geometric positioning according to the acquisition focal length label, the actual size label and the image size label to obtain an image acquisition characteristic distance. The step is to accurately calculate the distance information of the target object in the actual space by combining the focal length and the image size information, ensure the accuracy of the depth information and improve the positioning accuracy. And acquiring a laser point cloud through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark. The accurate three-dimensional environment model is constructed in the step, rich space information and depth data are provided, and the environment perception capability of the system is enhanced; by means of the coordinate identification of the monocular sensor, the three-dimensional model of the positioning target can be accurately placed in a three-dimensional environment. And positioning the three-dimensional model of the positioning target at the three-dimensional environment model according to the monocular sensor coordinate mark by the image acquisition characteristic distance, and performing visual display through a user side. The three-dimensional model of the positioning target is fused with the three-dimensional environment model, and accurate positioning is realized through the coordinate identification and the characteristic distance; and through visual display, a user can intuitively see the positioning result.
In summary, the application combines the monocular vision ranging and the laser point cloud technology, the three-dimensional environment model constructed by the laser scanner provides accurate three-dimensional space information, simultaneously, the three-dimensional model of the positioning target is used, the two-dimensional projection is compared with the actually captured two-dimensional image, the geometric positioning is performed by utilizing the acquisition focal length, the actual size and the image size, and the characteristic distance of the image acquisition is calculated more accurately, so that the positioning target can be fused with the actual environment better, the precision of target identification and positioning is improved, the accuracy and reliability of space positioning are obviously improved, the complex or changing environment can be better adapted, and the environmental adaptability is enhanced.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
Drawings
Fig. 1 is a schematic flow chart of a spatial positioning method of monocular vision ranging and laser point cloud fusion provided by an embodiment of the application;
Fig. 2 is a schematic flow chart of obtaining a selected two-dimensional projection slice image in a spatial positioning method of monocular vision ranging and laser point cloud fusion provided by an embodiment of the present application;
Fig. 3 is a schematic flow chart of obtaining an image acquisition characteristic distance in a monocular vision ranging and laser point cloud fusion spatial positioning method according to an embodiment of the present application;
Fig. 4 is a schematic structural diagram of a spatial positioning system with monocular vision ranging and laser point cloud fusion according to an embodiment of the present application.
Reference numerals illustrate: a monocular ranging module 10, a two-dimensional projection module 20, a shape analysis module 30, a geometric positioning module 40, a three-dimensional environment model construction module 50, and a target positioning module 60.
Detailed Description
The embodiment of the application solves the technical problem that the monocular positioning technology lacks depth information and global environment information to cause the limitation of positioning accuracy and environmental adaptability by providing the monocular visual ranging and laser point cloud fusion space positioning method and system, combines the monocular visual ranging and laser point cloud technology, improves the accuracy and reliability of space positioning, and simultaneously fuses with the actual environment better and enhances the environmental adaptability.
An embodiment of the present application provides a spatial positioning method for combining monocular vision ranging with laser point cloud, as shown in fig. 1, where the method includes:
step S1: and receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag.
Specifically, the monocular sensor refers to an image pickup apparatus having only one lens, such as a general video camera or a camera, for capturing a two-dimensional image. The two-dimensional image is a planar image captured by a monocular sensor, and contains only width and height information of an object, and does not contain depth information. The first acquisition position tag is a specific position of the monocular sensor at the time of recording an image capturing, for example, a coordinate position of the camera in space. The capture focal length tab refers to the focal length of a lens used in image capture, typically in millimeters (mm), such as a 50mm lens. The image size label is the resolution or size of the recorded image, e.g., 1920x1080 pixels.
First, the monocular sensor is set at a known position and angle, and shooting is performed in alignment with the target object. During shooting, the coordinates and focal length of the camera are recorded to form a first acquisition position tag and an acquisition focal length tag. For example, if the camera is fixed to the robot, the position of the robot and the focal length setting of the camera are the values of these tags. Then, the resolution or size of the image, i.e., the image size label, is recorded. Through this step, the monocular sensor records important metadata while acquiring images, and the tags and images provide data basis for subsequent positioning calculation.
Step S2: and loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image.
Specifically, the positioning target three-dimensional model refers to a three-dimensional stereoscopic model of a target object, which contains structural and shape information of the target object in a three-dimensional space. For example, a three-dimensional model of an automobile includes the length, width, height, and shape of the automobile. Two-dimensional projection is the process of converting a three-dimensional model into a two-dimensional planar image, typically by geometric transformation. This step causes the three-dimensional model to appear on a two-dimensional plane. The two-dimensional projection slice image is a two-dimensional image projected from the three-dimensional model, and simulates an image taken by a camera from a specific position.
A three-dimensional model of the target object is loaded using Computer Aided Design (CAD) software or a three-dimensional modeling tool (e.g., blender, 3ds Max). The three-dimensional model may be obtained from a pre-built database or may be created by a three-dimensional scanner. The viewing angle of the projection is determined using the first acquisition position tag, i.e. the position of the camera when the image was taken. This viewing angle is the same as the viewing angle at the time of actual image acquisition. For example, if the camera position is in front of the car, the projection angle corresponds to a frontal view of the car. The three-dimensional model is then perspective projected from this view angle, and the three-dimensional model is converted into a projection image on a two-dimensional plane, i.e., a two-dimensional projection slice image, using a geometric transformation algorithm. For example, conversion is performed using projection functions in OpenGL or other graphics libraries. The generated two-dimensional projection slice image is a two-dimensional representation of the three-dimensional model at a particular viewing angle, which will be used for subsequent shape analysis and localization. For example, a frontal view of an automobile model shows details of the front face, lights, grille, etc. of the automobile.
Step S3: and carrying out shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image has an actual size label.
Specifically, shape analysis refers to analyzing the shape of objects in images to determine the degree of similarity and matching between the images. Selecting a two-dimensional projection slice image refers to selecting one of a plurality of possible projection slice images that best matches the actually acquired two-dimensional image. This image will be used for subsequent positioning calculations. The full-size label refers to full-size information representing an object in an image, such as a length, a width, a height, etc. of the object.
And comparing the two-dimensional projection slice image with the actually acquired two-dimensional image of the positioning target. And selecting the best matched two-dimensional projection slice images by calculating the similarity between the images, and distributing the actual size labels to the two-dimensional projection slice images. This tag may be determined based on the focal length of the camera, the resolution of the image, or the size of the known object. Through the step, the best matched two-dimensional projection slice image can be selected, accurate size information is provided for the two-dimensional projection slice image, a basis is provided for subsequent accurate positioning, positioning accuracy and robustness are ensured, and the performance of the whole positioning system is improved.
Step S4: and performing geometric positioning according to the acquisition focal length label, the actual size label and the image size label to obtain an image acquisition characteristic distance.
In particular, the image acquisition feature distance refers to the actual distance between the object in the image and the camera, which is calculated geometrically. The acquisition focal length tag, the image size tag in step S1, and the actual size tag in step S3 are read. A neural network model is trained through historical data, the acquisition focal length label, the actual size label and the image size label are input into the neural network model for geometric positioning, the model can analyze according to the labels, and the image acquisition characteristic distance is output. Through the step, the actual position and the size of the object in the image can be determined, and accurate space positioning is realized.
Step S5: and acquiring a laser point cloud through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark.
Specifically, a laser scanner is a sensor that emits laser light and measures the reflected laser light to create a three-dimensional model of an object or environment. A laser point cloud is a collection of a large number of points generated by a laser scanner representing a three-dimensional structure of an object or environment. The three-dimensional environmental model is a three-dimensional representation of an environment constructed based on laser point cloud data, including all objects in the environment and their spatial relationships. The monocular sensor coordinate identification specifies the position and orientation of the monocular sensor in the three-dimensional environmental model.
A laser scanner is used to emit laser light and record the time or phase change of the laser light reflected back from the object to determine the distance and angle of the object. These data are used to create a laser point cloud, i.e. a three-dimensional environment model. In building the model, the position and orientation of the monocular sensor in the environment is recorded, and this information is used as a coordinate identification. This process may be implemented using laser scanning software (e.g., rhinoceros or CloudCompare). Through the step, an accurate three-dimensional environment model can be created, the method is suitable for complex and changeable environment conditions, high-precision positioning can be still kept under various illumination and texture conditions, and meanwhile, the position of the monocular sensor is accurately marked so as to be used in the subsequent positioning and data fusion processes.
Step S6: and positioning the three-dimensional model of the positioning target at the three-dimensional environment model according to the monocular sensor coordinate mark by the image acquisition characteristic distance, and performing visual display through a user side.
Specifically, a three-dimensional model of a positioning target is imported into three-dimensional modeling software, and the position of the three-dimensional model is converted into a coordinate system of a three-dimensional environment model according to monocular sensor coordinate identification. And accurately placing the three-dimensional model of the positioning target at a corresponding position in the three-dimensional environment model according to the image acquisition characteristic distance. And then, the visual display is carried out through the user side, so that the user can see the accurate position of the target object in the environment. The user side may be a computer screen or a mobile device, etc. This process may be implemented using three-dimensional modeling software (e.g., unity or Unreal Engine). Through the step, the target object can be accurately positioned in a three-dimensional environment, and visual display is carried out through the user side, so that visual navigation and positioning information is provided for the user.
Further, step S2 of the embodiment of the present application further includes:
step S21: the first acquisition position label comprises a lens deflection angle label, wherein the lens deflection angle label refers to an included angle between a lens of the monocular sensor and the horizontal direction.
Step S22: and setting the lens deflection angle label as image incidence angle information, wherein the image incidence angle information refers to an included angle between a connecting line of the first acquisition position and the positioning target and the horizontal direction.
Step S23: and carrying out two-dimensional projection on the three-dimensional model of the positioning target according to the image incidence angle information to obtain the two-dimensional projection slice image.
Specifically, the lens deflection angle label refers to the angle of a lens of a monocular sensor (such as a camera) relative to the horizontal direction, and describes the inclination degree of the lens. It affects the viewing angle of the image and the way in which objects are projected in the image. The image incidence angle information refers to the angle of light entering the camera lens, i.e. the angle between the line connecting the camera to the target object and the horizontal direction. This angle is critical to understanding the position and orientation of the object in the image.
First, the tilt angle of the camera is determined using the lens deflection angle label. This angle is used as image incidence angle information, i.e. to determine the orientation of the camera relative to the target object. This angle of incidence information is then used to adjust the viewing angle of the three-dimensional model, ensuring that the projection of the model coincides with the viewing angle in the actual image. Then, perspective projection is performed on the adjusted three-dimensional model, and a two-dimensional projection slice image is generated. This process is typically implemented on a computer using three-dimensional rendering and projection algorithms, such as using three-dimensional modeling software or projective transformation functions in a computer vision library. Through the step, the system can accurately simulate the visual angle of a camera, generate a two-dimensional projection slice image matched with an actual acquired image, and provide accurate reference for subsequent positioning and fusion.
Further, as shown in fig. 2, step S3 of the embodiment of the present application further includes:
Step S31: and configuring a first alignment angle point and a second alignment angle point of the two-dimensional projection slice image based on the image incidence angle information, wherein the first alignment angle point is positioned at a first incidence position preset azimuth, and the second alignment angle point is positioned at a second incidence position preset azimuth.
Step S32: and configuring a third alignment corner point and a fourth alignment corner point of the positioning target two-dimensional image based on the lens deflection angle label, wherein the third alignment corner point is positioned at a preset azimuth of a first incidence position, and the fourth alignment corner point is positioned at a preset azimuth of a second incidence position.
Step S33: and after aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain the selected two-dimensional projection slice image.
In particular, the alignment corner points refer to specific points on the image for aligning and matching two images. The first and second alignment angular points are positioned on the two-dimensional projection slice image, and the third and fourth alignment angular points are positioned on the two-dimensional image of the positioning target. The first and second predetermined locations of incidence refer to specific locations in the image that are predefined for assisting in aligning the image. These orientations may be specific points in the image, such as corners of a building or edges of a landmark.
First, according to the image incidence angle information, one specific position on the two-dimensional projection slice image is determined as a first alignment corner point, and the other specific position is determined as a second alignment corner point. The angular points are positioned in preset positions, the position of the first alignment angular point is the preset position of the first incidence position, and the position of the second alignment angular point is the preset position of the second incidence position. Then, a third alignment corner point and a fourth alignment corner point in the two-dimensional image of the positioning target are determined by using the lens deflection angle label, and the corner points are also positioned in a preset azimuth. The third alignment angular point is located at the first incidence position preset azimuth, and the fourth alignment angular point is located at the second incidence position preset azimuth. Then, the third alignment corner is aligned with the first alignment corner and the fourth alignment corner is aligned with the second pair Ji Jiaodian to ensure that the two images are consistent at these key points. This alignment process may be accomplished using image registration techniques, such as using feature matching algorithms or rigid body transformations to adjust the image to match at these points. After alignment, shape analysis is performed, the best matching two-dimensional projection slice image is selected, and a full-size label is assigned to the best matching two-dimensional projection slice image. The steps can accurately align and match the images, ensure that the selected two-dimensional projection slice image is consistent with the actually acquired image on key points, and further ensure that the selected two-dimensional projection slice image is consistent with the acquired positioning target two-dimensional image.
Preferably, step S33 of the embodiment of the present application further includes:
step S331: and after aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape similarity analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a shape similarity evaluation result.
Step S332: and when the shape similarity evaluation result is greater than or equal to a shape similarity threshold, setting the two-dimensional projection slice image as the selected two-dimensional projection slice image.
Step S333: and when the shape similarity evaluation result is smaller than the shape similarity threshold value, carrying out two-dimensional projection on the three-dimensional model of the positioning target based on the image incidence angle information, and updating the two-dimensional projection slice image.
Specifically, shape similarity analysis refers to calculating a similarity score between two images by comparing shape features in the images, such as edges, contours, corner points, etc. The shape similarity evaluation result is an output result of the shape similarity analysis, typically a numerical value, representing the degree of similarity of the two images. The shape similarity threshold is a similarity score criterion set in advance for judging whether the two images are sufficiently similar.
And after aligning the third alignment angular point and the first alignment angular point as well as the fourth alignment angular point and the second alignment angular point, carrying out shape similarity analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a shape similarity evaluation result. This process may be implemented by edge detection, contour extraction, feature point matching, and other techniques. And then, a similarity threshold value is customized in advance, the shape similarity evaluation result is compared with the shape similarity threshold value, and if the shape similarity evaluation result is larger than or equal to the preset shape similarity threshold value, the current two-dimensional projection slice image is set as the selected two-dimensional projection slice image. If the evaluation result is smaller than the threshold value, the matching is insufficient, two-dimensional projection is conducted on the three-dimensional model of the positioning target again based on the image incidence angle information, and a new two-dimensional projection slice image is generated to find a better matching. The two-dimensional projection slice image selected through the step has high similarity in shape with the actually acquired image, so that the consistency of the three-dimensional model and an actual scene is ensured, an accurate reference is provided for the subsequent accurate positioning, and the positioning accuracy and the positioning robustness are improved.
Illustratively, the vehicle is spatially localized and a front view image of the vehicle is captured by a monocular sensor. A front view image projected from a three-dimensional automobile model. The two-dimensional projection slice image of the automobile model is aligned with the positioning target two-dimensional image according to the pair Ji Jiaodian, namely the upper left corner of the automobile image is aligned with the upper left corner of the projection slice image, and the lower right corner of the automobile image is aligned with the lower right corner of the projection slice image. The Canny edge detection algorithm of OpenCV is used to extract edges in the image. Contours in the image are extracted using a contour detection algorithm. And performing feature point matching by using a SIFT feature point detection and matching algorithm. Then, a similarity score is calculated using the number or distance of the matched feature points. The shape similarity threshold was set to 0.8. And if the similarity score is 0.85, the condition is met, and the image is selected.
Further, as shown in fig. 3, step S4 of the embodiment of the present application further includes:
Step S41: the acquisition focal length dataset, the actual size dataset, and the image size dataset are configured by a monocular sensor model.
Step S42: and traversing the acquired focal length data set, the actual size data set and the image size data set to calibrate the sample distance, and acquiring a historical image acquired sample distance data set.
Step S43: and monitoring the output of the BP neural network by using the historical image acquisition sample distance data set, and configuring a characteristic distance estimator by using the acquisition focal length data set, the actual size data set and the image size data set as inputs.
Step S44: and geometrically positioning the acquisition focal length tag, the actual size tag and the image size tag according to the characteristic distance estimator to obtain the image acquisition characteristic distance.
Specifically, the monocular sensor model refers to the specific model of monocular camera used. The acquired focal length data set comprises a plurality of data sets with different focal length values, and shooting information under different focal length conditions is recorded. The full-size dataset comprises a plurality of datasets of full sizes of different objects for calibration and calculation. The image size dataset comprises a plurality of image datasets of different resolutions or sizes. Sample distance calibration refers to determining the actual distance under different parameter combinations through experimental data, and is used for creating reference data. The historical image acquisition sample distance dataset is a collection that records all distance data obtained through sample distance calibration. The characteristic distance estimator is a tool for predicting and estimating the characteristic distance of image acquisition by using a neural network and historical data.
First, the acquisition focal length dataset, the actual size dataset, and the image size dataset are configured according to the monocular sensor model. These datasets contain image acquisition samples at different focal lengths and object sizes. Then, the datasets are traversed for sample distance calibration, i.e., the camera is calibrated using objects of known distance to establish a relationship between focal length, actual size, and image size, generating a historical image acquisition sample distance dataset. Next, a BP neural network model is constructed using a machine learning framework, such as TensorFlow or PyTorch, and the BP neural network is trained using these sample data sets. Taking an acquisition focal length data set, an actual size data set and an image size data set as model input, taking a historical image acquisition sample distance data set to monitor the output of the BP neural network, adjusting weights and offsets through a back propagation algorithm until the model converges, and configuring the trained BP neural network as a characteristic distance estimator. The evaluator uses a BP neural network to geometrically locate the acquisition focal length tag, the full-size tag, and the image-size tag to calculate the image acquisition feature distance. The geometric positioning accuracy is improved by the aid of a machine learning method, and the calculation of the image acquisition characteristic distance is ensured to be more accurate.
Preferably, step S42 of the embodiment of the present application further includes:
Step S421: and extracting first acquisition focal length data, first actual size data and first image size data according to the acquisition focal length data set, the actual size data set and the image size data set.
Step S422: and taking the first acquisition focal length data, the first actual size data and the first image size data as limitations, and acquiring a historical sample of the monocular sensor model to obtain a historical sample record distance data set.
Step S423: and carrying out centralized number analysis according to the historical sample record distance data set to obtain first historical image acquisition sample distance data, and adding the first historical image acquisition sample distance data into the historical image acquisition sample distance data set.
Specifically, a specific focal length value is randomly selected from the acquired focal length data set as the first acquired focal length data. A specific one of the real size data sets is randomly selected as the first real size data. A specific image size data is randomly selected from the image size data set as the first actual size data. And taking the extracted first acquisition focal length data, first actual size data and first image size data as conditions, and carrying out image acquisition on an object with a known distance by using a monocular sensor with a corresponding model to obtain a history sample record distance data set so as to establish the relation among focal length, actual size and image size. Wherein the historical sample record distance data set is a sample distance data set recorded according to a specific condition. Next, the system performs a central numerical analysis on the historical sample record distance dataset, calculates a central tendency, such as an average or median, of the data, obtains first historical image collection sample distance data, and adds it to the historical image collection sample distance dataset. Repeating the steps to obtain a plurality of historical image acquisition sample distance data, and adding the data into a historical image acquisition sample distance data set to provide an accurate data basis for subsequent neural network training.
For example, sample data collection and analysis is performed on an automobile model using a brand of monocular sensor model. The following data are extracted from the acquisition focal length dataset, the full-size dataset, and the image-size dataset: first acquisition focal length data: 50mm; first actual size data: an automobile 1.5 meters high; first image size data: 1920x1080 pixels. According to the extracted data, sample data acquisition is carried out by using the corresponding monocular sensor model, and a historical sample record distance data set corresponding to the focal length, the actual size and the measured distance is obtained: {10.1 meters, 10.2 meters, 10.0 meters, 10.3 meters, 10.1 meters }. And carrying out centralized number analysis on the historical sample record distance data set to obtain first historical image acquisition sample distance data. The average value of the data set was calculated to give a result of 10.14 m. The calculated average distance of 10.14 meters is added to the historical image acquisition sample distance dataset.
Further, step S6 of the embodiment of the present application further includes:
Step S61: and performing geometric positioning analysis on the three-dimensional environment model based on the monocular sensor coordinate identification and the image acquisition characteristic distance according to the lens deflection angle label to obtain a target space coordinate.
Step S62: and acquiring the three-dimensional model attitude data of the positioning target according to the acquisition azimuth of the selected two-dimensional projection slice image.
Step S63: and deploying the positioning target three-dimensional model on the three-dimensional environment model according to the positioning target three-dimensional model posture data by using the target space coordinates.
Specifically, the target space coordinates refer to specific position coordinates of the target object in the three-dimensional environment model. The positioning target three-dimensional model posture data is data describing a specific posture of the positioning target three-dimensional model, such as a direction, an inclination angle, and the like.
Firstly, according to the lens deflection angle label, geometric positioning analysis is carried out in a three-dimensional environment model by using monocular sensor coordinate identification and image acquisition characteristic distance, and a triangulation or perspective projection technology can be used to obtain the space coordinates of a target object. Then, the attitude data of the target object, namely the direction and angle of the object, are determined according to the acquisition azimuth of the selected two-dimensional projection slice image. Finally, the three-dimensional model of the positioning target is accurately placed in the three-dimensional environment model by using the space coordinates and the gesture data of the target.
Illustratively, the monocular sensor lens deflection angle is obtained to be-20 degrees, geometric positioning analysis is carried out by using the coordinates (1,2,1.5) of the sensor and the image acquisition characteristic distance of 10.14 meters, and the target space coordinates (10.613, -1.475,1.5) are calculated. And then, calculating to obtain the attitude data of the three-dimensional model of the positioning target as-20 degrees according to the acquisition azimuth-20 degrees of the selected two-dimensional projection slice image. Finally, the automobile three-dimensional model is positioned to target space coordinates (10.613-1.475,1.5), and is rotated by-20 degrees according to the calculated gesture data, accurately deployed in the three-dimensional environment model, and visually displayed through a user side to provide global positioning information.
In summary, the spatial positioning method of monocular vision ranging and laser point cloud fusion provided by the embodiment of the application has the following technical effects:
And receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag. These tags provide the shooting location of the image, the focal length of the camera, and the size information of the image, providing the basis data for subsequent positioning calculations. Loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image; this step converts the three-dimensional information into a two-dimensional image, generating a slice image that coincides with the actual scene for comparison with the image acquired by the monocular sensor. And carrying out shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image has an actual size label. This step ensures that the selected two-dimensional projection slice image is exactly matched with the actual image by shape analysis, providing reliable dimensional information and providing an accurate data basis for geometric positioning. And performing geometric positioning according to the acquisition focal length label, the actual size label and the image size label to obtain an image acquisition characteristic distance. The step is to accurately calculate the distance information of the target object in the actual space by combining the focal length and the image size information, ensure the accuracy of the depth information and improve the positioning accuracy. And acquiring a laser point cloud through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark. The step builds an accurate three-dimensional environment model, provides rich space information and depth data, enhances the environment perception capability of the system, and can adapt to complex and changeable environment conditions. And positioning the three-dimensional model of the positioning target at the three-dimensional environment model according to the monocular sensor coordinate mark by the image acquisition characteristic distance, and performing visual display through a user side. According to the method, the attitude data of the target three-dimensional model is calculated through geometric positioning analysis and acquisition azimuth of the two-dimensional projection slice image, so that accurate placement and attitude adjustment of the target object in a three-dimensional environment are realized, and the authenticity and accuracy of positioning are ensured; and through visual display, a user can intuitively see the positioning result.
In the embodiment of the application, the monocular vision ranging and the laser point cloud technology are combined, the three-dimensional environment model constructed by the laser scanner provides accurate three-dimensional space information, meanwhile, the three-dimensional model of the positioning target is used, the two-dimensional projection is compared with the actually captured two-dimensional image, the geometric positioning is carried out by utilizing the acquisition focal length, the actual size and the image size, the characteristic distance of the image acquisition is calculated more accurately, the positioning target can be fused with the actual environment better, the target identification and positioning precision is improved, the space positioning accuracy and reliability are obviously improved, the complex or changing environment can be better adapted, and the environmental adaptability is enhanced.
In a second embodiment, as shown in fig. 4, an embodiment of the present application provides a spatial positioning system for combining monocular vision ranging with a laser point cloud, where the system includes:
The monocular distance measurement module 10 is used for receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position label, an acquisition focal length label and an image size label.
And the two-dimensional projection module 20 is used for loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image.
The shape analysis module 30 is configured to perform shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, where the selected two-dimensional projection slice image has a full-size label.
The geometric positioning module 40 is configured to perform geometric positioning according to the acquisition focal length tag, the actual size tag, and the image size tag, so as to obtain an image acquisition feature distance.
The three-dimensional environment model construction module 50 is used for constructing a three-dimensional environment model by collecting laser point clouds through a laser scanner, wherein the three-dimensional environment model has monocular sensor coordinate identification.
And the target positioning module 60 is used for positioning the three-dimensional model of the positioning target to the three-dimensional environment model according to the monocular sensor coordinate identification and the image acquisition characteristic distance, and performing visual display through a user side.
Further, the two-dimensional projection module 20 according to the embodiment of the present application is further configured to perform the following steps:
the first acquisition position tag comprises a lens deflection angle tag, wherein the lens deflection angle tag refers to an included angle between a lens of the monocular sensor and the horizontal direction; setting the lens deflection angle label as image incidence angle information, wherein the image incidence angle information refers to an included angle between a connecting line of a first acquisition position and a positioning target and a horizontal direction; and carrying out two-dimensional projection on the three-dimensional model of the positioning target according to the image incidence angle information to obtain the two-dimensional projection slice image.
Further, the shape analysis module 30 according to the embodiment of the present application is further configured to perform the following steps:
Configuring a first alignment angle point and a second alignment angle point of the two-dimensional projection slice image based on the image incidence angle information, wherein the first alignment angle point is positioned at a first incidence position preset azimuth, and the second alignment angle point is positioned at a second incidence position preset azimuth; based on the lens deflection angle label, configuring a third alignment angle point and a fourth alignment angle point of the positioning target two-dimensional image, wherein the third alignment angle point is positioned at a preset azimuth of a first incidence position, and the fourth alignment angle point is positioned at a preset azimuth of a second incidence position; and after aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain the selected two-dimensional projection slice image.
Further, the shape analysis module 30 according to the embodiment of the present application is further configured to perform the following steps:
After aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape similarity analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a shape similarity evaluation result; when the shape similarity evaluation result is greater than or equal to a shape similarity threshold, setting the two-dimensional projection slice image as the selected two-dimensional projection slice image; and when the shape similarity evaluation result is smaller than the shape similarity threshold value, carrying out two-dimensional projection on the three-dimensional model of the positioning target based on the image incidence angle information, and updating the two-dimensional projection slice image.
Further, the geometric positioning module 40 according to the embodiment of the present application is further configured to perform the following steps:
Configuring and collecting a focal length data set, an actual size data set and an image size data set through a monocular sensor model; performing sample distance calibration by traversing the acquisition focal length data set, the actual size data set and the image size data set to obtain a historical image acquisition sample distance data set; monitoring the output of the BP neural network by using the historical image acquisition sample distance data set, and configuring a characteristic distance estimator by using the acquisition focal length data set, the actual size data set and the image size data set as inputs; and geometrically positioning the acquisition focal length tag, the actual size tag and the image size tag according to the characteristic distance estimator to obtain the image acquisition characteristic distance.
Further, the geometric positioning module 40 according to the embodiment of the present application is further configured to perform the following steps:
Extracting first acquisition focal length data, first actual size data and first image size data according to the acquisition focal length data set, the actual size data set and the image size data set; taking the first acquisition focal length data, the first actual size data and the first image size data as limitations, and acquiring a historical sample of the monocular sensor model to obtain a historical sample record distance data set; and carrying out centralized number analysis according to the historical sample record distance data set to obtain first historical image acquisition sample distance data, and adding the first historical image acquisition sample distance data into the historical image acquisition sample distance data set.
Further, the object positioning module 60 according to the embodiment of the present application is further configured to perform the following steps:
Performing geometric positioning analysis on the three-dimensional environment model based on the monocular sensor coordinate identification and the image acquisition characteristic distance according to the lens deflection angle label to obtain a target space coordinate; acquiring three-dimensional model attitude data of a positioning target according to the acquisition azimuth of the selected two-dimensional projection slice image; and deploying the positioning target three-dimensional model on the three-dimensional environment model according to the positioning target three-dimensional model posture data by using the target space coordinates.
Through the foregoing detailed description of the spatial positioning method of monocular vision ranging and laser point cloud fusion, those skilled in the art can clearly know that the spatial positioning system of monocular vision ranging and laser point cloud fusion in this embodiment, for the system disclosed in the second embodiment, since it corresponds to the method disclosed in the first embodiment, it has corresponding functional modules and beneficial effects, and relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. The spatial positioning method for fusion of monocular vision ranging and laser point cloud is characterized by comprising the following steps:
Receiving a positioning target two-dimensional image through a monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag, the first acquisition position tag is a specific position of the monocular sensor when recording an image shooting, the acquisition focal length tag is a lens focal length used when the image shooting, and the image size tag is resolution or size of the recording image;
Loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image;
Performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image is provided with an actual size label, and the actual size label refers to actual size information representing an object in the image;
Performing geometric positioning according to the acquisition focal length tag, the actual size tag and the image size tag to obtain an image acquisition characteristic distance, wherein the image acquisition characteristic distance refers to the actual distance between an object in an image and a camera;
Acquiring laser point clouds through a laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark, and the monocular sensor coordinate mark designates the position and the direction of a monocular sensor in the three-dimensional environment model;
and positioning the three-dimensional model of the positioning target at the three-dimensional environment model according to the monocular sensor coordinate mark by the image acquisition characteristic distance, and performing visual display through a user side.
2. The method for spatially locating a fusion of monocular vision ranging and laser point cloud as claimed in claim 1, wherein loading a three-dimensional model of a locating target, performing two-dimensional projection according to the first acquisition position tag, and obtaining a two-dimensional projection slice image, comprises:
the first acquisition position tag comprises a lens deflection angle tag, wherein the lens deflection angle tag refers to an included angle between a lens of the monocular sensor and the horizontal direction;
Setting the lens deflection angle label as image incidence angle information, wherein the image incidence angle information refers to an included angle between a connecting line of a first acquisition position and a positioning target and a horizontal direction;
And carrying out two-dimensional projection on the three-dimensional model of the positioning target according to the image incidence angle information to obtain the two-dimensional projection slice image.
3. The method of spatial localization of a monocular vision ranging fused with a laser point cloud of claim 2, wherein performing a shape analysis on the two-dimensional projection slice image and the localization target two-dimensional image to obtain a selected two-dimensional projection slice image comprises:
Configuring a first alignment angle point and a second alignment angle point of the two-dimensional projection slice image based on the image incidence angle information, wherein the first alignment angle point is positioned at a first incidence position preset azimuth, and the second alignment angle point is positioned at a second incidence position preset azimuth;
Based on the lens deflection angle label, configuring a third alignment angle point and a fourth alignment angle point of the positioning target two-dimensional image, wherein the third alignment angle point is positioned at a preset azimuth of a first incidence position, and the fourth alignment angle point is positioned at a preset azimuth of a second incidence position;
And after aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain the selected two-dimensional projection slice image.
4. A monocular vision ranging and laser point cloud fusion spatial positioning method according to claim 3, wherein after aligning the third alignment corner and the first alignment corner, and aligning the fourth alignment corner and the second alignment corner, performing shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain the selected two-dimensional projection slice image, comprising:
after aligning the third alignment angular point and the first alignment angular point and the fourth alignment angular point and the second alignment angular point, performing shape similarity analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a shape similarity evaluation result;
When the shape similarity evaluation result is greater than or equal to a shape similarity threshold, setting the two-dimensional projection slice image as the selected two-dimensional projection slice image;
and when the shape similarity evaluation result is smaller than the shape similarity threshold value, carrying out two-dimensional projection on the three-dimensional model of the positioning target based on the image incidence angle information, and updating the two-dimensional projection slice image.
5. The method for spatially locating a fusion of monocular vision ranging and laser point cloud as defined in claim 1, wherein geometrically locating according to the acquisition focal length tag, the actual size tag and the image size tag to obtain an image acquisition feature distance comprises:
configuring and collecting a focal length data set, an actual size data set and an image size data set through a monocular sensor model;
Performing sample distance calibration by traversing the acquisition focal length data set, the actual size data set and the image size data set to obtain a historical image acquisition sample distance data set;
monitoring the output of the BP neural network by using the historical image acquisition sample distance data set, and configuring a characteristic distance estimator by using the acquisition focal length data set, the actual size data set and the image size data set as inputs;
and geometrically positioning the acquisition focal length tag, the actual size tag and the image size tag according to the characteristic distance estimator to obtain the image acquisition characteristic distance.
6. The method for spatially locating a fusion of monocular vision ranging and laser point cloud of claim 5, wherein performing sample distance calibration by traversing the acquisition focal length dataset, the actual size dataset, and the image size dataset to obtain a historical image acquisition sample distance dataset, comprising:
Extracting first acquisition focal length data, first actual size data and first image size data according to the acquisition focal length data set, the actual size data set and the image size data set;
Taking the first acquisition focal length data, the first actual size data and the first image size data as limitations, and acquiring a historical sample of the monocular sensor model to obtain a historical sample record distance data set;
and carrying out centralized number analysis according to the historical sample record distance data set to obtain first historical image acquisition sample distance data, and adding the first historical image acquisition sample distance data into the historical image acquisition sample distance data set.
7. The method for spatially locating a fusion of monocular vision ranging and laser point cloud as claimed in claim 2, wherein locating the three-dimensional model of the locating target at the three-dimensional environmental model by the image acquisition feature distance according to the monocular sensor coordinate identification comprises:
performing geometric positioning analysis on the three-dimensional environment model based on the monocular sensor coordinate identification and the image acquisition characteristic distance according to the lens deflection angle label to obtain a target space coordinate;
acquiring three-dimensional model attitude data of a positioning target according to the acquisition azimuth of the selected two-dimensional projection slice image;
And deploying the positioning target three-dimensional model on the three-dimensional environment model according to the positioning target three-dimensional model posture data by using the target space coordinates.
8. A monocular vision ranging and laser point cloud fusion spatial positioning system, characterized in that the system is configured to perform the monocular vision ranging and laser point cloud fusion spatial positioning method according to any one of claims 1 to 7, and comprises:
The monocular distance measurement module is used for receiving a positioning target two-dimensional image through the monocular sensor, wherein the positioning target two-dimensional image is provided with a first acquisition position tag, an acquisition focal length tag and an image size tag, the first acquisition position tag is a specific position of the monocular sensor when recording image shooting, the acquisition focal length tag refers to a lens focal length used when the image shooting, and the image size tag is resolution or size of the recording image;
the two-dimensional projection module is used for loading a positioning target three-dimensional model, and carrying out two-dimensional projection according to the first acquisition position label to obtain a two-dimensional projection slice image;
The shape analysis module is used for carrying out shape analysis on the two-dimensional projection slice image and the positioning target two-dimensional image to obtain a selected two-dimensional projection slice image, wherein the selected two-dimensional projection slice image is provided with an actual size label, and the actual size label refers to actual size information representing an object in the image;
The geometric positioning module is used for performing geometric positioning according to the acquisition focal length tag, the actual size tag and the image size tag to obtain an image acquisition characteristic distance, wherein the image acquisition characteristic distance refers to the actual distance between an object in an image and a camera;
The three-dimensional environment model construction module is used for acquiring laser point clouds through the laser scanner to construct a three-dimensional environment model, wherein the three-dimensional environment model is provided with a monocular sensor coordinate mark, and the monocular sensor coordinate mark designates the position and the direction of a monocular sensor in the three-dimensional environment model;
And the target positioning module is used for positioning the three-dimensional model of the positioning target to the three-dimensional environment model according to the monocular sensor coordinate identification and the image acquisition characteristic distance, and performing visual display through a user side.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411124770.7A CN118642121B (en) | 2024-08-16 | 2024-08-16 | Monocular vision ranging and laser point cloud fusion space positioning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411124770.7A CN118642121B (en) | 2024-08-16 | 2024-08-16 | Monocular vision ranging and laser point cloud fusion space positioning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118642121A CN118642121A (en) | 2024-09-13 |
| CN118642121B true CN118642121B (en) | 2024-11-19 |
Family
ID=92665232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411124770.7A Active CN118642121B (en) | 2024-08-16 | 2024-08-16 | Monocular vision ranging and laser point cloud fusion space positioning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118642121B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106441151A (en) * | 2016-09-30 | 2017-02-22 | 中国科学院光电技术研究所 | Measuring system for three-dimensional target Euclidean space reconstruction based on vision and active optical fusion |
| CN108932475A (en) * | 2018-05-31 | 2018-12-04 | 中国科学院西安光学精密机械研究所 | Three-dimensional target identification system and method based on laser radar and monocular vision |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110827415B (en) * | 2019-11-11 | 2022-08-23 | 吉林大学 | All-weather unknown environment unmanned autonomous working platform |
| CN113108773A (en) * | 2021-04-22 | 2021-07-13 | 哈尔滨理工大学 | Grid map construction method integrating laser and visual sensor |
| CN117152240A (en) * | 2022-05-19 | 2023-12-01 | 鸿富锦精密电子(烟台)有限公司 | Object detection method, device, equipment and storage medium based on monocular camera |
| CN117409393A (en) * | 2023-11-02 | 2024-01-16 | 北京同创信通科技有限公司 | Method and system for detecting laser point cloud and visual fusion obstacle of coke oven locomotive |
-
2024
- 2024-08-16 CN CN202411124770.7A patent/CN118642121B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106441151A (en) * | 2016-09-30 | 2017-02-22 | 中国科学院光电技术研究所 | Measuring system for three-dimensional target Euclidean space reconstruction based on vision and active optical fusion |
| CN108932475A (en) * | 2018-05-31 | 2018-12-04 | 中国科学院西安光学精密机械研究所 | Three-dimensional target identification system and method based on laser radar and monocular vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118642121A (en) | 2024-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| EP2111530B1 (en) | Automatic stereo measurement of a point of interest in a scene | |
| JP4677536B1 (en) | 3D object recognition apparatus and 3D object recognition method | |
| Yan et al. | Joint camera intrinsic and lidar-camera extrinsic calibration | |
| US8792726B2 (en) | Geometric feature extracting device, geometric feature extracting method, storage medium, three-dimensional measurement apparatus, and object recognition apparatus | |
| JP2004127239A (en) | Method and system for calibrating multiple cameras using calibration object | |
| CN112254670B (en) | 3D information acquisition equipment based on optical scanning and intelligent vision integration | |
| Phuc Truong et al. | Registration of RGB and thermal point clouds generated by structure from motion | |
| JP2016217941A (en) | Three-dimensional evaluation device, three-dimensional data measurement system and three-dimensional measurement method | |
| CN114140539B (en) | Method and device for acquiring position of indoor object | |
| CN113096183A (en) | Obstacle detection and measurement method based on laser radar and monocular camera | |
| CN114413958A (en) | Monocular visual ranging and speed measurement method for unmanned logistics vehicles | |
| CN114137564A (en) | Automatic indoor object identification and positioning method and device | |
| KR100574227B1 (en) | Apparatus and method for extracting object motion that compensates for camera movement | |
| Furukawa et al. | Interactive shape acquisition using marker attached laser projector | |
| CN113269729B (en) | Assembly body multi-view detection method and system based on depth image contrast | |
| CN112082486B (en) | Handheld intelligent 3D information acquisition equipment | |
| CN112253913B (en) | Intelligent visual 3D information acquisition equipment deviating from rotation center | |
| CN118642121B (en) | Monocular vision ranging and laser point cloud fusion space positioning method and system | |
| KR102065337B1 (en) | Apparatus and method for measuring movement information of an object using a cross-ratio | |
| CN115601438A (en) | External parameter calibration method and device and autonomous mobile equipment | |
| CN112254671B (en) | Multi-time combined 3D acquisition system and method | |
| CN112254679A (en) | Multi-position combined 3D acquisition system and method | |
| CN112254677A (en) | Multi-position combined 3D acquisition system and method based on handheld device | |
| Uyanik et al. | A method for determining 3D surface points of objects by a single camera and rotary stage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |