CN116124150B

CN116124150B - A method for indoor multi-mobile robot positioning based on camera splicing and area matching

Info

Publication number: CN116124150B
Application number: CN202310195421.3A
Authority: CN
Inventors: 梁忠超; 王艳锋; 黄茁
Original assignee: Northeastern University China
Current assignee: Northeastern University China
Priority date: 2023-03-02
Filing date: 2023-03-02
Publication date: 2025-06-06
Anticipated expiration: 2043-03-02
Also published as: CN116124150A

Abstract

The present invention provides a method for indoor multi-mobile robot positioning with camera splicing and area matching, comprising the following steps: firstly, a coordinate conversion model is established, parameters are obtained through camera calibration, and the pixel coordinates of the mobile robot in the pixel coordinate system are converted into world coordinates in the world coordinate system; a label is set on the mobile robot, and the outline of the label in each frame image under the field of view angle is obtained through a joint algorithm of related image processing; finally, the minimum bounding box of each outline is obtained through function calculation, and the area of the label on each mobile robot can be obtained, and the function of multi-target matching can be realized with different areas; at the same time, the center of mass position can be obtained, and finally the orientation and attitude angle of each mobile robot can be obtained. In the case of indoor positioning, the problem of small monocular camera screen is solved by fixing multiple cameras and using the screen splicing algorithm. The present invention solves the problem of multi-robot positioning difficulties, has high positioning accuracy, and reduces the complexity of visual positioning.

Description

Indoor multi-mobile robot positioning method with camera splicing and area matching functions

Technical Field

The invention relates to an indoor multi-mobile robot vision positioning method, belongs to the technical field of vision robots, and particularly relates to a multi-mobile robot vision positioning method with multiple fixed cameras.

Background

The positioning technology is one of key technologies for mobile robot research, and plays a decisive role in normal operation of the robot. The positioning problem of the robot has very important significance in the intelligent research of the mobile robot, and is a key for realizing autonomous navigation and completing complex intelligent tasks in a specific environment.

Indoor positioning of a mobile robot refers to that the robot senses information of surrounding environment through a sensor and performs analysis processing, and the position information and the posture information of the robot are solved by using a specific positioning algorithm. At present, the application of visual positioning in the field of robot positioning is more and more extensive, and the visual positioning is the hottest direction in the current mobile robot indoor positioning technology research by virtue of the advantages of rich information acquisition, strong understanding capability to environment, lasting stability, low cost and the like.

The existing indoor visual positioning method still has defects in the actual positioning process and is easily limited by the field angle of a single camera, and has the problem that the positioning method is relatively complex, for example, the CN210119230U patent discloses an indoor visual positioning system which mainly uses an image acquisition device and a monocular camera to position a target so as to achieve the visual positioning effect, but the positioning method uses a monochromatic illumination module to acquire the image position so as to realize the positioning of the target, the positioning method is relatively complex, and the CN209280914U patent discloses an indoor visual positioning system which mainly uses an optical signal to acquire the image so as to achieve the indoor positioning effect, but the positioning method adopts an LED lamp to transmit the signal so as to acquire the image so as to realize the positioning of the target, and the positioning method is relatively complex.

Disclosure of Invention

In order to overcome the defects of the background technology, the invention provides a camera splicing and area matching indoor multi-mobile robot positioning method

The invention adopts the following technical means:

The indoor multi-mobile robot positioning method with the camera splicing and area matching comprises the following steps:

calibrating a monocular camera to obtain camera parameters, calibrating a ground map, and establishing a one-to-one correspondence between a ground map area and a real ground map area in a video image acquired by the monocular camera;

The method comprises the steps of 2, tracking a plurality of robots through a positioning tracking algorithm, fixing markers on a mobile robot, simultaneously using the markers as labels, matching different areas of each label with corresponding robots, and tracking the plurality of robots through tracking the markers with different areas;

step 3, positioning a centroid point, replacing a robot position coordinate with the centroid coordinate of the marker, acquiring the vertex coordinate and the three-side length of the marker through a two-point positioning pose, sequencing the three-side lengths, and acquiring the shortest length of the three-side lengths;

and 4, obtaining a moving track image of the mobile robot in a mode of splicing pictures of the cameras by fixing the cameras, so as to identify and position the moving track image.

Compared with the prior art, the invention has the following advantages:

The invention uses labels with different areas to well play a role in matching, and different mobile robots are distinguished; the method can simultaneously position a plurality of mobile robots, improves the positioning time and efficiency, and solves the problem of low positioning precision of the robots.

According to the invention, the bottom edge of each tag is shortest, and the head and the tail of each mobile robot can be well determined, so that the attitude angle in any direction can be obtained.

According to the invention, a plurality of fixed cameras are used for splicing a plurality of obtained pictures, so that the problem that a single camera is smaller in display picture is effectively solved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort to a person skilled in the art.

FIG. 1 is a flow chart of the overall algorithm of the present invention.

FIG. 2 is a schematic diagram of a filtering algorithm according to the present invention.

FIG. 3 is a schematic diagram of the world, camera, image and pixel coordinate systems of the present invention.

Fig. 4 is a schematic diagram of the multi-camera fixing of the present invention (taking an experimental field covered by the camera 1 as an example).

FIG. 5 is a diagram of the experimental position of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

As shown in fig. 1-5, the invention provides a positioning method of an indoor multi-mobile robot with camera splicing and area matching, which comprises the following steps:

The method comprises the steps of 1, calibrating a monocular camera to obtain camera parameters, calibrating a ground map, establishing a one-to-one correspondence between a ground map area and a real ground map area in a video image acquired by the monocular camera, establishing a geometric model of camera imaging for determining the correlation between the position of a space object and a corresponding point in the image, and establishing a coordinate conversion relation according to the camera imaging model. In the conversion process of the coordinate system, the conversion relation between the world coordinate system and the pixel coordinate system is as follows:

Wherein [ X _w Y_w Z_w 1]^T ] represents the corresponding point homogeneous coordinates of the world coordinate system, [ X _c Y_c Z_c 1]^T ] represents any point homogeneous coordinates of the camera coordinate system, [ u v 1] ^T represents the corresponding points in the pixel coordinate system; The unit is pixel, f represents camera focal length, and unit is millimeter, d _x、d_y represents physical dimension of each pixel on x axis and y axis, and unit is millimeter; Representing the internal parameters of the camera, Representing camera outliers, R representing the rotation matrix, and t representing the translation vector.

The method comprises the steps of 2, tracking a plurality of robots through a positioning tracking algorithm, fixing markers on a mobile robot, simultaneously using the markers as labels, matching different areas of each label with corresponding robots, tracking the plurality of robots through tracking the markers with different areas, obtaining a color space of each frame of picture through a camera, converting RGB into LAB, graying, performing Gaussian filtering noise reduction and image corrosion operation, filtering red in the LAB color space, finding out the outline of each label, outputting the outline, and finally, traversing each outline, obtaining the area of each label through a function, and placing the area into a list for sorting, thereby realizing matching tracking.

The method comprises the steps of 3, positioning a centroid point, replacing a robot position coordinate with the centroid coordinate of a marker, obtaining the vertex coordinate of the marker, the length of three sides and sequencing the lengths of the three sides by two-point positioning gestures, obtaining the shortest length of the three sides by the three-point positioning gestures, obtaining the length of each side and sequencing by an identification algorithm, obtaining the shortest side as a bottom side, determining two vertex coordinates of the bottom side, obtaining the center coordinate of the bottom side according to the two vertex coordinates of the bottom side, and obtaining the gesture angle of the robot by combining the centroid coordinates;

Step 4, fixing a plurality of cameras around the field to obtain a moving track image of the mobile robot in a mode of splicing pictures of the cameras, converting each frame of picture in the video image into a required color space, graying, performing Gaussian filtering to reduce noise and image corrosion to enable the outline of the label to be more obvious, traversing the outlines of all labels and calculating the areas of the labels, and finally combining the method in the step 3, so that identification and positioning are performed. .

In a preferred embodiment, in the present application, the step 1 further includes the steps of:

Step 1-1, camera calibration and de-distortion, namely converting the world coordinate system into a pixel coordinate system, so that the positions of corresponding points on the surface of an object in space correspond to the positions of each point in an image one by one, calibrating the camera by using a Zhang Zhengyou calibration method to obtain internal parameters of the camera And Ginseng radix

The distortion process comprises radial distortion and tangential distortion, wherein the radial distortion is distortion distributed along the radial direction of the lens, the tangential distortion is generated due to the fact that the lens is not parallel to the plane of a camera sensor or the plane of an image, the influence is small, only the radial distortion is considered, the distortion parameter k ₁,k₂ is obtained by combining the internal parameter, the external parameter and the distortion model of the camera, and the distortion model is:

Where (u, v) denotes the ideal undistorted pixel coordinates, Representing the pixel coordinates after distortion, in pixels,(X, y) represents the ideal undistorted successive image coordinates,Representing the successive image coordinates after distortion in millimeters and k ₁,k₂ representing the distortion parameters.

Step 1-2, selecting an initial calibration point and an expected calibration point;

And 1-3, carrying out multi-picture inverse perspective transformation and splicing to obtain an image with the shape consistent with that of the ground map, wherein the spliced image after the multi-picture inverse perspective transformation is obtained. In the multi-picture inverse perspective transformation and splicing process, after each frame of picture of each camera is de-distorted, respectively adopting an inverse perspective transformation method to splice multiple pictures of an experimental field, and then utilizing the scaling relationship between the spliced pixel point coordinates and a real ground map to obtain the position coordinates of each robot under the real ground map, wherein the i-th camera inverse perspective transformation formula is as follows:

Wherein [ X _i y_i 1]^T ] represents the pixel coordinate system point coordinates of the original image of the ith camera, [ X _i Y_i Z_i]^T ] represents the pixel point coordinates of the corresponding point after transformation of the ith camera, M _i represents the transformation matrix of the ith camera of 3×3, and since the pixel coordinate system is two-dimensional coordinates, the transformation is two-dimensional to three-dimensional transformation, and the transformed pixel point coordinates are transformed to the corresponding point coordinates on the two-dimensional coordinate system [ X _i′Y_i′Z_i′]^T:

From the above, four pairs of coordinate values are needed to solve all unknowns in the M _i matrix, where (L _i,h_i)、(0,h_i)、(0,0)、(l_i, 0) represents the initial coordinate point of the original image of the ith camera, and (L _i,H_i)、(0,H_i)、(0,0)、(L_i, 0) represents the desired coordinate point after reverse perspective of the ith camera, all in pixels (piexl). The scaling of the spliced pixel coordinates and the real ground map in length: scaling of the spliced pixel coordinates and the real ground map in width: wherein L and H respectively represent the length and width of the real ground map;

Therefore, after the n cameras are spliced, the position coordinates of each robot under the real ground map are as follows: Wherein X represents the abscissa of each robot under the real ground map, Y represents the ordinate of each robot under the real ground map, and the units are meters.

In order to verify the effectiveness of the indoor multi-mobile robot positioning method with camera splicing and area matching in practical engineering application, the positioning effect is tested from the aspect of static positioning precision based on a self-built experimental platform.

In the positioning precision experiment, 9 positions shown in the figure in the scene range are found, the mobile robot is respectively placed at 9 points for positioning, the statistical algorithm outputs coordinate data, and the coordinate data is compared with the data measured in the actual experiment for analysis. The average static errors obtained by the test are all within 5mm, and the positioning accuracy is high.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments. In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners.

It should be noted that the above embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solution described in the above embodiments may be modified or some or all of the technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present invention.

Claims

1. A method for indoor multi-mobile robot positioning with camera splicing and area matching, characterized in that it includes the following steps:

Step 1, calibrate the monocular camera to obtain camera parameters; at the same time, calibrate the ground map to establish a one-to-one correspondence between the ground map area in the video image captured by the monocular camera and the real ground map area;

Step 2: Track multiple robots through positioning and tracking algorithms; fix markers on the mobile robots and use the markers as labels. The different areas of each label match the corresponding robots. Track multiple robots by tracking markers of different areas.

Step 3: Perform centroid positioning, and replace the robot position coordinates with the centroid coordinates of the marker; locate the posture through two points, the two points represent the centroid coordinates and the shortest side center coordinates of the marker respectively, obtain the vertex coordinates and three side lengths of the marker and sort the three side lengths to obtain the shortest length of the three side lengths; obtain the three vertex coordinates of the marker through the recognition algorithm, thereby obtaining the length of each side and sorting them, and the shortest side is the bottom side, and determine the two vertex coordinates of the bottom side; then obtain the center coordinates of the bottom side according to the two vertex coordinates of the bottom side, and obtain the posture angle of the robot in combination with the centroid coordinates;

Step 4: By fixing multiple cameras around the site, the activity trajectory image of the mobile robot is obtained in the form of splicing multiple camera images, and each frame of the video image is converted into the required color space and grayed out; Gaussian filtering and image erosion are then performed to make the outline of the label more obvious; then the outlines of all labels are traversed and their areas are calculated; finally, the method in step 3 is combined to perform identification and positioning.

2. The method for indoor multi-mobile robot positioning with camera splicing and area matching according to claim 1, characterized in that the step 1 further comprises the following steps:

Step 1-1: Camera calibration and distortion removal;

Step 1-2: Selection of initial calibration points and expected calibration points;

Step 1-3: Inverse perspective transformation and stitching of multiple images, obtaining a stitched image after inverse perspective transformation of multiple images that is consistent with the shape of the ground map.

3. According to the camera splicing and area matching indoor multi-mobile robot positioning method described in claim 1, it is characterized in that in order to determine the one-to-one correspondence between the position of the spatial object and its corresponding point in the image mentioned in step 1, a camera imaging geometric model is established; based on the camera imaging model, a coordinate transformation relationship is established.

4. According to the method for indoor multi-mobile robot positioning with camera splicing and area matching in claim 3, it is characterized in that, in the camera imaging geometric model, during the conversion process of the coordinate system, the conversion relationship between the world coordinate system and the pixel coordinate system is:

Among them, [X _w Y _w Z _w 1] ^T represents the homogeneous coordinates of the corresponding point in the world coordinate system; [X _c Y _c Z _c 1] ^T represents the homogeneous coordinates of any point in the camera coordinate system; [uv 1] ^T represents the corresponding point in the pixel coordinate system; are called the normalized focal lengths of the camera on the x-axis and y-axis, respectively, in pixels; f represents the focal length of the camera, in millimeters; _dx and _dy represent the physical size of each pixel on the x-axis and y-axis, respectively, in millimeters; represents the camera internal parameters, represents the camera external parameters, R represents the rotation matrix, and t represents the translation vector.

5. According to the method for indoor multi-mobile robot positioning with camera splicing and area matching described in claim 2, the camera calibration and dedistortion process, that is, the process of converting from the world coordinate system to the pixel coordinate system, makes the position of the corresponding point on the surface of the object in space correspond to the position of each point in the image one by one; the camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera's internal parameters and external reference

The distortion process includes radial distortion and tangential distortion. The radial distortion is the distortion distributed along the radius of the lens. The tangential distortion is caused by the non-parallelism between the lens itself and the camera sensor plane or the image plane, and has a small impact. Therefore, only radial distortion is considered. The distortion parameters k ₁ and k ₂ are obtained by combining the camera internal and external parameters and the distortion model. The distortion model is:

Among them, (u, v) represents the ideal undistorted pixel coordinates, Represents the pixel coordinates after distortion, in pixels. (x,y) represents the ideal undistorted continuous image coordinates, represents the coordinates of the continuous image after distortion, in millimeters; k ₁ , k ₂ represent the distortion parameters.

6. According to claim 2, a method for indoor multi-mobile robot positioning with camera splicing and area matching is characterized in that, in the process of inverse perspective transformation and splicing of multiple images, after each frame of each camera is dedistorted, the multi-image splicing of the experimental site is performed by inverse perspective transformation, and then the scaling relationship between the pixel coordinates after splicing and the real ground map is used to obtain the position coordinates of each robot under the real ground map; the formula for inverse perspective transformation of the i-th camera is as follows:

Among them, _[xiyi1 _] ^T represents the coordinates of the original image pixel coordinate system of _the i-th camera, [ _XiYiZi ] ^T represents the pixel coordinates of the corresponding point after the i-th camera transformation, and _Mi represents the 3×3 transformation matrix of the i-th camera. Since the pixel coordinate system is a two _- dimensional coordinate system, and this transformation is a two-dimensional to three-dimensional conversion, the transformed pixel coordinates are converted to the corresponding point coordinates in the _two -dimensional coordinate system [ _Xi′Yi′Zi _′ ] ^T :

From the above formula, we can get that four pairs of coordinate values are needed to solve all the unknowns in the _Mi matrix, where ( _li , _hi ), (0, _hi ), (0,0), ( _li ,0) represent the initial calibration points of the original image of the i-th camera, and ( _Li , _Hi ), (0, _Hi ), (0,0), ( _Li ,0) represent the expected calibration points after the i-th camera inverse perspective, and the units are all pixels (piexl); then the scaling ratio of the length of the spliced pixel coordinates and the real ground map is: The scaling ratio of the pixel coordinates after stitching to the width of the real ground map: Among them, L and H represent the length and width of the real ground map respectively;

Therefore, after the n cameras are spliced together, the position coordinates of each robot on the real ground map are: Among them, X represents the horizontal coordinate of each robot under the real ground map, and Y represents the vertical coordinate of each robot under the real ground map, and the units are both meters.

7. According to the method for indoor multi-mobile robot positioning with camera splicing and area matching described in claim 1, it is characterized in that in the step 2, the color space of each frame of the picture obtained by the camera is converted from RGB to LAB and grayed; then Gaussian filtering and image corrosion operations are performed; then the red color is filtered in the LAB color space, the outline of each label is found and output; finally, after traversing each outline, the area of each label is obtained by using a function and put into a list for sorting, thereby realizing matching tracking.