CN114549978A - A method and system for running a mobile robot based on multiple cameras - Google Patents

Abstract

A mobile robot operation method based on multiple cameras is characterized in that multiple cameras acquire indoor scene images, coordinate systems are established according to two sides of the scene images, and a scene image area of each camera is set; setting a rectangular boundary area in a scene image, establishing a target image template, matching the target image template with the scene image by using a gray matching algorithm to obtain the center point coordinate and the contour of the target image in the scene image, and determining the coordinate of the target image in a coordinate system according to the center point coordinate and the contour of the target image. According to the multi-camera-based mobile robot operation method, the image template is matched with the scene image, the gray level matching algorithm is used for matching, the matching precision under various conditions can be effectively improved, the accuracy of object positioning is improved, the hardware cost is reduced, the reliability of a positioning system is improved by using a variable and quantitative method, and the center point coordinate and the contour coordinate of the target image are accurately obtained.

Description

A method and system for operating a mobile robot based on multiple cameras

technical field

The invention relates to the technical field of indoor positioning of sweeping robots, in particular to a multi-camera-based mobile robot operation method and system.

Background technique

The existing sweeping robots clean indoors and have a great effect. With the change of usage scenarios, it is urgent to accurately position the sweeping robot indoors to solve the operations of navigation, cleaning and obstacle avoidance.

However, in the image data of the existing sweeping robot, the color easily overlaps with other approximations of the indoor scene, resulting in difficult calibration and accurate tracking.

At present, the main method is to locate by comparing static and dynamic objects. The requirements and costs for computer computing power and system algorithm design are relatively high, and it will be affected by the environment, such as the reflection of product mirrors, etc. The reason for this problem is that Improvements based on current hardware have led to the tendency to use algorithms to locate moving objects, ignoring the conversion between variables and quantities.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the problems existing in the prior art, and to provide an operation method and system for a multi-camera mobile robot with high calibration accuracy and low cost.

The technical scheme of the present invention is as follows:

A mobile robot operation method based on multi-camera,

Multiple cameras acquire indoor scene images, establish a coordinate system according to both sides of the scene image, and set the scene image area of each camera;

Sets the rectangular bounding area in the scene image,

Establish a target image template, match the target image template with the scene image using a grayscale matching algorithm, and obtain the center point coordinates and outline of the target image in the scene image,

Determine the coordinates of the target image in the coordinate system according to the center point coordinates and outline of the target image.

After the setting of the rectangular boundary area in the scene image, it further includes establishing an image shielding area for the scene image, and the image shielding area includes

Process the area outside the rectangular bounding area in the scene image;

The scene image in this area is grayscaled and XORed with the constant 255.

When the multiple cameras capture images, the image capturing area of each camera is set, and the camera is designated for the scene image according to the image capturing area partition definition.

Taking the two sides of the rectangular boundary as axes, a coordinate system is established, and the coordinates of the position of the target image within the rectangular boundary are calibrated.

An independent coordinate system is set for the scene image of each camera.

The coordinate systems in the scene images of the adjacent cameras are connected to the coordinate systems of the scene images of the first camera, and a plurality of cameras form a panoramic coordinate system.

The scene image is obtained by taking a downward image of the camera set at the top of the room, and the image capturing area of the multiple cameras covers the entire indoor scene.

Analyze the scene images of consecutive cameras, and mark the robot as lost when the robot leaves the rectangular area in the scene image and does not enter the rectangular area of the next camera.

The image template includes a plurality of pixels, and the pixels of each image template are from the target image pixel normally collected to the minimum recognition pixel of the target image.

A mobile robot operating system based on multiple cameras, including

The camera module is used for the camera to obtain indoor scene images and establish a coordinate system;

Template building blocks, build image templates,

The matching module matches the image template and the scene image using a grayscale matching algorithm, and normalizes the cross-correlation value to obtain a matching result;

The calculation module calculates the center point coordinates and contour coordinates of the target image in the scene image.

Beneficial effects:

A mobile robot operation method based on multi-camera,

Multiple cameras acquire indoor scene images, establish a coordinate system according to both sides of the scene image, and set the scene image area of each camera;

Sets the rectangular bounding area in the scene image,

Establish a target image template, match the target image template with the scene image using a grayscale matching algorithm, and obtain the center point coordinates and outline of the target image in the scene image,

Determine the coordinates of the target image in the coordinate system according to the center point coordinates and outline of the target image.

The multi-camera-based mobile robot operation method of the present invention uses an image template to match a scene image, and uses a grayscale matching algorithm for matching, which can effectively improve the matching accuracy in various situations and improve the accuracy of object positioning. Reduce hardware cost, use variable and quantitative methods to improve the reliability of the positioning system, and accurately obtain the center point coordinates and contour coordinates of the target image.

Description of drawings

Fig. 1 is the flow chart of the target image template of the present invention;

FIG. 2 is a schematic diagram of the rectangular boundary, contour and motion trend area of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application d or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

Example 1

A mobile robot operation method based on multi-camera,

Multiple cameras acquire indoor scene images, establish a coordinate system according to both sides of the scene image, and set the scene image area of each camera;

Sets the rectangular bounding area in the scene image,

Establish a target image template, match the target image template with the scene image using a grayscale matching algorithm, and obtain the center point coordinates and outline of the target image in the scene image,

Determine the coordinates of the target image in the coordinate system according to the center point coordinates and outline of the target image.

The center point coordinates and outline of the target image in the obtained scene image include:

Match the target image template to the scene image in the rectangular boundary area, and determine the contour of the target image in the scene image,

store the coordinates of the bounding area and outline of the rectangle,

The scene image of the present invention is obtained by the following method:

Control the camera to obtain indoor scene images, including one or more cameras to capture images of indoor scenes respectively. When the indoor scene is large, multiple cameras need to be used to capture images of indoor scenes. The number of cameras is related to the indoor scene. The range of the image is proportional; the control camera to obtain the indoor scene image is the camera set at the top of the room to take pictures downward to obtain the indoor scene image; the time to obtain the indoor scene image is the minimum time for the robot to move away from the motion trend area. time.

The multiple cameras are respectively set with an image capturing angle and an image capturing area, so that they can fully cover the indoor scene, wherein the image capturing area of the multiple cameras is defined, and the image of the intersecting area in the scene image is determined. The designated camera, that is, when the target image is located in each imaging area, calculates the data in the image according to the data of the corresponding camera.

As shown in FIG. 1 , the method for creating a target image template of the present invention includes filtering a scene image into a black-and-white photo through grayscale processing, extracting a template of the target image, and then obtaining an image template.

The pixel size of the target image template includes the minimum recognized pixels of the target image to the pixels of the complete target image; multiple target image templates can be set, and each can be set from the minimum recognized pixel to the pixel size of the complete target image. can effectively improve the overall matching success rate.

Of course, the pixel size of the recognition can also be adjusted according to the difference between the target images; at the same time, the position of the target image template is also meaningful, and the target images in the target image template are respectively located on two diagonal lines in the scene image.

The objects in the scene image of the present application are all expressed by a coordinate system, and the coordinate system takes the two sides of the scene image as the coordinate axes, that is, the X-axis and the Y-axis, and uses each pixel as a coordinate point (x, y) to form a coordinate system, which is Objects of interest in the scene image are marked with coordinates so that they can be used quickly.

The coordinate system in this application can also be set through a rectangular boundary, that is, a certain point on the boundary of the rectangular boundary is used as the starting element point, and the two sides are used as the coordinate axes to establish a coordinate system. When the scene image needs to be calculated, it can be directly Match the scene images within the rectangle boundary without analyzing and comparing the images outside the rectangle boundary.

The coordinate system of the present invention can also be formed in other ways, the coordinate systems in the scene images of the adjacent cameras are connected to the coordinate system of the scene image of the first camera, and the scene images of multiple cameras form a panoramic coordinate system.

When the scene image of the present application is processed, it will encounter the problem of low efficiency. This is because the scene image is a wide-angle image, which contains a lot of link noise, such as walls, decorations, etc., depending on the complexity of the room. A large variable, which will reduce the recognition efficiency and increase the probability of misjudgment.

As shown in FIG. 2 , the present invention can utilize rectangular boundaries and contours to achieve fast processing of scene images. First, a rectangular boundary is set to exclude certain parts of the scene image, such as fixing objects that cannot exist, and then a rectangular boundary is set. Match the target image template internally, find the target image in the scene image, and determine the center coordinate point and outline coordinates of the target image.

The range of this rectangular boundary can be designed as needed. It can not only remove environmental noise, but also can be set as a specific recognition area. The rectangular boundary can use coordinates to determine an area. When the scene image needs to be recognized, the rectangular boundary is automatically taken. .

The coordinate system in this application can also be set through a rectangular boundary, that is, a certain point on the boundary of the rectangular boundary is used as the starting element point, and the two sides are used as the coordinate axes to establish a coordinate system. When the scene image needs to be calculated, it can be directly Match the scene images within the rectangle boundary without analyzing and comparing the images outside the rectangle boundary.

The pixels of the scene image outside the rectangular boundary are processed using the image masking method, and an image masking area is established for the scene image, and the image masking area includes the area outside the rectangular boundary area in the processing scene image; XOR with the constant 255; 255 represents black, and all places outside the box selection are black.

In addition to the rectangular boundary and outline, the motion trend area can also be set. Specifically, matching the scene image to the template formed by the target image refers to comparing the template formed by the image with the scene image to identify the target image in the scene image. Determine the contour, and determine the motion trend area according to the contour, and the area of the rectangle is larger than the area of the target image.

When the next frame of scene image matching fails to find the target image, all the scene images of this frame are matched, and the contour and motion trend area are re-determined.

The motion trend area includes a horizontal rectangular trend and a vertical rectangular trend. The horizontal trend area and the rectangular trend area start from the left border of the scene image and end to the right, and the vertical rectangular trend starts from the upper border of the scene image and goes to the bottom. So far; the width of the horizontal rectangular trend and the vertical rectangular trend is greater than the width of the outline.

The motion trend area can be used to predict the motion position of the robot and track and locate it. At the same time, when recognizing scene images, the pixels in the motion trend area can be preferentially matched to match the target image; it can not only improve the matching efficiency, It can also achieve better tracking effect.

The motion trend area realizes the tracking and positioning of the target image in the scene image, and effectively reduces the matching time of the scene image and the amount of matching calculation.

The scene image of the present invention also includes image preprocessing, and the image preprocessing refers to adjusting the parameters in the scene image: that is, red, yellow, blue, hue, saturation, brightness, brightness and other parameters, and converting the scene image into a single parameter Image. The scene image is matched against the template image using this single parameter. The camera in this application uses a CMOS camera, and its connector uses an interface: USB2.0; pixels: 2 million pixels (effective pixels: 1.2-1.5 million pixels) frame rate: 30; sensitivity: 120-150LUX.

The scene image obtained in this application can also be subjected to obstacle identification processing, which includes acquiring an initial image of the scene image, comparing it with the actual scene image to obtain a difference, and identifying the obstacle area in the scene image; The images are compared to find the difference, the corresponding obstacle area is extracted, and the obstacle reduction map is generated; the actual coverage area is determined according to the cleaning area and the obstacle reduction map, and the coverage rate is determined according to the actual coverage area.

The trajectory determined according to the first position and the second position is that the line connecting the coordinates of the center point of the first position and the coordinates of the center point of the second position is the trajectory line; the coordinates of the rectangular boundary in the two adjacent scene images system connection.

The template formed by the described matching target image, the template image is filtered into a black and white photo, the degree of black and white is quantified into a numerical value (0-255), and the described grayscale matching method is to normalize between the template image and the target image. method of calculation

For example: when the template image T(x,y) of size L×K moves from top to bottom and left to right in the target image f(x,y) of N×M (L≦M and K≦N) , the cross-correlation value of the region where (i,j) (0≦i≦M-1; 0≦j≦N-1) is located in the template image and the target image can be expressed as:

The cross-correlation value is very sensitive to changes in amplitude in the image and template, such as pixel density, so normalization is needed to eliminate the effect of amplitude:

where is the average density value of the pixels in the template w; in simple terms, normalization is to convert all values to 0 and 1 by setting a threshold. After normalization, the recognition speed can be improved. Correspondingly, the false positive rate is also will increase, so the present invention normalizes the matching success rate. If the matching success rate is lower than 100%, it is judged that the matching is unsuccessful, and only when it reaches 100%, it is judged that the matching is successful.

The matching rate can also be improved by the following methods,

In the environment of the present invention, the brightness of the indoor lighting lamps is uniformly set, which can effectively improve the matching efficiency.

Preferably, a marker can also be superimposed on the target image, and the marker is a combination of one or more elements selected from non-reflective materials, single color, and symmetrical pattern, so that the target image can be more easily matched.

The final acceptance form of the target detection of moving objects is to output the center of the moving object on the screen and identify the center of the moving object. First, the coordinates of the center point of the matching moving object will be output in the matching area and stored through the data structure cluster. The coordinates of the center point, and then output the values of the coordinates x and y to the contour parameter Contours cluster in the ROI through an array, and save it, which needs to convert the point into the storage form of the ROI, cover it and output it to the ROI. can be output.

First, two parameters need to be output, which are the number of successfully matched images and the coordinates of the successfully matched points; only when the number of successfully matched images is greater than zero, the current point coordinates of the moving object will be input.

Those of ordinary skill in the art can understand that the realization of all or part of the processes in the methods of the above embodiments can be accomplished by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the program is During execution, it may include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

It should be understood that although the various steps in the flowchart of the accompanying drawings are sequentially shown in the order indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order and may be performed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution sequence is also It does not have to be performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of sub-steps or stages of other steps.

A mobile robot operating system based on multiple cameras, comprising a camera module for acquiring indoor scene images and establishing a coordinate system; a template building module for establishing an image template, and a matching module for using a grayscale matching algorithm between the image template and the scene image Matching is performed, and the cross-correlation value is normalized to obtain a matching result; the calculation module calculates the center point coordinates and contour coordinates of the target image in the scene image.

The present application also provides another implementation manner, which is to provide a computer-readable storage medium, where the computer-readable storage medium stores a multi-camera-based mobile robot running program, and the multi-camera-based mobile robot The running program can be executed by at least one processor, so that the at least one processor executes the steps of the above-mentioned method for running a mobile robot based on multiple cameras.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims (10)

1. a mobile robot operating method based on multiple cameras, is characterized in that: Multiple cameras acquire indoor scene images, establish a coordinate system according to both sides of the scene image, and set the scene image area of each camera; Sets the rectangular bounding area in the scene image, Establish a target image template, match the target image template with the scene image using a grayscale matching algorithm, and obtain the center point coordinates and outline of the target image in the scene image, Determine the coordinates of the target image in the coordinate system according to the center point coordinates and outline of the target image. 2 . The multi-camera-based mobile robot operation method according to claim 1 , wherein the setting of the rectangular boundary area in the scene image further comprises establishing an image shielding area for the scene image, and the image shielding Areas include: Process the area outside the rectangular bounding area in the scene image; The scene image in this area is grayscaled and XORed with the constant 255. 3. A kind of mobile robot operation method based on multi-camera according to claim 1, it is characterized in that: when the said multiple cameras capture images, the image capturing area of each camera is set, and the scene image is captured according to the image capturing area. Partition definition, specifying the camera. 4 . The multi-camera-based mobile robot operation method according to claim 2 , wherein the two sides of the rectangular boundary are used as axes to establish a coordinate system to calibrate the coordinates of the position of the target image within the rectangular boundary. 5 . 5 . The method for running a mobile robot based on multiple cameras according to claim 1 , wherein an independent coordinate system is set for the scene image of each camera. 6 . 6. The method for running a mobile robot based on multiple cameras according to claim 1, wherein the coordinate system in the scene image of the adjacent cameras is connected to the coordinate system of the scene image of the first camera, and the coordinate systems in the scene images of the adjacent cameras are connected. The cameras form a panoramic coordinate system. 7. a kind of mobile robot operation method based on multi-camera according to claim 1, is characterized in that: obtain scene image by the camera that is arranged on the indoor top to take images downward, and the imaging area of a plurality of cameras is covered with whole indoor Scenes. 8. a kind of mobile robot operation method based on multi-camera according to claim 1, is characterized in that: analyze the scene image of continuous camera, when the robot leaves the rectangular area in the scene image, and does not enter the rectangle of the next camera When in an area, the marked robot is lost. 9. The multi-camera-based mobile robot operation method according to claim 1, wherein the image template includes a plurality of pixels, and the pixels of each image template are the difference between the target image pixel of the normal collection and the target image pixel. Minimum recognition pixel. 10. a mobile robot operating system based on multiple cameras, is characterized in that: comprising: The camera module is used for the camera to obtain indoor scene images and establish a coordinate system; Template building blocks, build image templates, The matching module matches the image template and the scene image using a grayscale matching algorithm, and normalizes the cross-correlation value to obtain a matching result; The calculation module calculates the center point coordinates and contour coordinates of the target image in the scene image.

Images