CN116012443A - A Pose Measurement Method for On-orbit Satellites Based on Parallelogram Fitting - Google Patents

Abstract

The invention provides an on-orbit satellite pose measurement method based on parallelogram fitting, which comprises the following steps: step A, collecting an image; step B, designing an image processing algorithm, and extracting the outline in the image processing algorithm; step C, combining contour points according to the set judging conditions; step D, selecting a target contour; e, performing parallelogram fitting on the target contour points by utilizing a parallelogram fitting algorithm; and F, extracting corner points and resolving the pose of the target satellite. The beneficial effects of the invention are as follows: 1. the on-orbit Wei Xingwei pose measurement method is applicable to a plurality of occasions and has high fitting precision; 2. the on-orbit Wei Xingwei pose measurement method disclosed by the invention gradually approaches the best fit line in an iterative mode, can eliminate the interference of points with large errors, and is applicable to both closed contour points and discrete point sets.

Description

A Pose Measurement Method for On-orbit Satellites Based on Parallelogram Fitting

technical field

The invention relates to the field of aviation technology, in particular to an in-orbit satellite pose measurement method based on parallelogram fitting.

Background technique

With the increase of space activities in various countries, space missions are becoming more and more complex, and tasks such as satellite search, detection, approach, and accompanying flight are increasing, and visual detection is an important method. Therefore, it is necessary to develop high-efficiency and high-precision visual measurement algorithms .

Contents of the invention

In order to solve the problems in the prior art, the present invention provides an on-orbit satellite pose measurement method based on parallelogram fitting.

The invention provides a method for measuring the position and attitude of satellites in orbit based on parallelogram fitting, comprising the following steps:

Step A, collecting images.

Step B, designing an image processing algorithm to extract the contours therein.

Step C, combining the contour points according to the set judgment conditions.

Step D, target contour selection.

Step E, using a parallelogram fitting algorithm to perform parallelogram fitting on the target contour points.

Step F, extract corner points and calculate the pose of the target satellite.

As a further improvement of the present invention, in the step B, it also includes:

Step B1, image distortion removal.

Step B2, according to the area of the target satellite in the image, set the ROI search area.

Step B3, converting the image into an HLS format suitable for machine judgment.

Step B4, performing median filtering on the image to reduce noise interference.

In step B5, the target area is extracted according to the HLS value of each pixel of the image, and the image is converted into a binary image.

Step B6, use the morphological closing operation to reduce the interference in the image, fill the gaps in the target area, and reduce the extraction error.

Step B7, extract the contour.

As a further improvement of the present invention, in the step C, it also includes:

Step C1, use the minimum enveloping rectangle to fit each contour point set, and obtain the center point, side length, and rotation angle around the center point of the fitted rectangle.

Step C2, starting from the first contour, sequentially judge the relationship between the distance between the two contours and the side length of the contour fitting rectangle, if the difference is greater than the threshold, merge the contour point set into the previous contour point set, and delete the latter one Set of contour points.

Step C3, return to step C1, repeat steps C2 and C3 until no contours can be merged.

Step C4, filter the contour point set according to the number of points in each contour point set, and delete the contour with a small number of contour points.

As a further improvement of the present invention, in the step D, the extraction target is the contour corresponding to the solar panel, including:

In step D1, the contour with the largest number of contour points is selected as the contour of the target solar panel.

Step D2, distinguish the outline of the sailboard according to the positional relationship of the sailboard on the image plane. Among the center point coordinates (u, v), the sailboard with the smallest u is the leftmost sailboard, the one with u in the middle is the middle sailboard, and the one with the largest u is the farthest sailboard. Right side sailboard.

As a further improvement of the present invention, in the step E, it also includes:

Step E1, obtain the contour point set to be fitted, and calculate the center point of the contour, that is, the average value of the (u, v) coordinates of each point.

Step E2, using the least squares method to obtain an optimal fitting straight line ax+by+c=0, the straight line passing through the center point of the contour point set.

Step E3, determining point sets fitting two edge lines of a parallelogram approximately parallel to the optimal straight line.

Step E4, use the least squares method to fit the best fitting straight lines of the upper and lower contour point sets respectively, the two fitted straight lines need to be parallel straight lines, use the two contour point sets to fit the two straight lines respectively, and then calculate the deflection angle of the straight line , calculate the weighted average of the deflection angle of the straight line according to the number of elements in the two contour point sets, and then determine the a, b, c1, c2 of the two straight lines.

Step E5, determining point sets for fitting the other two parallel lines of the parallelogram.

Step E6, fitting another two parallel straight lines according to the point set obtained in step E5.

Step E7, according to the preliminarily determined point set for fitting the four sides of the parallelogram and the preliminarily confirmed straight line equations of the four sides, gradually approach the optimal straight line equation of the four sides of the parallelogram in an iterative manner.

As a further improvement of the present invention, in the step E3, it also includes:

Step E30 , according to the positional relationship between the contour points and the straight line, the contour points are divided into two parts: points above the straight line (including on the straight line) and points below the straight line.

Step E31, calculate the distance from each contour point to the straight line, and find the maximum distances from the points above the straight line and the points below the straight line to the straight line respectively.

Step E32, for the points above and below the straight line, assign them to different point sets according to the difference between the distance from each point to the straight line and the maximum distance obtained in step E31.

Step E33, for the upper and lower points, if the number of elements in the farthest point set is the largest, or the number of elements in the farthest point set is greater than the threshold, then the farthest point set is the set where the contour edge points are located, otherwise the remaining point set has the largest number of elements The point set is the set where the contour edge points are located.

As a further improvement of the present invention, in the step E5, it also includes:

Step E50, finding the point set in step E4 whose distance from the two fitted straight lines is less than the threshold and on the edge of the point set as the starting search point for the other two sides of the parallelogram to be fitted.

Step E51, solve the straight line passing through two points through the two pairs of initial search points obtained in step E50, use these two straight lines as the two initial sidelines of the parallelogram to be fitted, and search for points whose distance from the straight line is smaller than the threshold value as the fitted line. Combine the point sets of the two sidelines of the parallelogram to fit the straight line formed by the two part point sets, the method is the same as step E4.

As a further improvement of the present invention, in the step E7, it also includes:

Step E70, for the point set used to fit the four sides of the parallelogram, determine the distance between each point and the fitted straight line, if it is less than the threshold, keep it, and if it is greater than the threshold, delete it.

Step E71, using the point set processed in step E70 to fit two pairs of parallel edge lines of a parallelogram, the method is the same as step E4.

Step E72, judging whether the points deleted twice before and after are smaller than the threshold, if yes, then end, if not, then return to step E70, wherein the distance threshold used in step E70 will decrease with the increase of the number of iterations.

As a further improvement of the present invention, in the step F, it also includes:

Step F1, solving the four corner points formed by the intersection of the four fitted straight lines.

Step F2, find the pose of the satellite in the camera coordinate system.

As a further improvement of the present invention, in the step F2, if the three-dimensional size of the solar panel is known, the P12P algorithm can be used for the monocular camera, and if a certain corner point has a large fitting error, the corner point can be excluded, Using the remaining corner fitting, for a binocular camera or a multi-eye camera, you can use the P12P algorithm to solve the satellite pose for each monocular camera image and take the average value. If you don’t know the three-dimensional size of the solar panel, you can use a binocular camera Or a multi-eye camera, after solving the position of each corner point, the pose of the satellite is obtained.

The beneficial effects of the present invention are: 1. The on-orbit satellite pose measurement method of the present invention is suitable for many occasions, and the fitting accuracy is high; 2. The on-orbit satellite pose and pose measurement method of the present invention gradually approaches the best fitting by iterative mode Lines can eliminate the interference of points with large errors, and are applicable to both closed contour points and discrete point sets.

Description of drawings

Fig. 1 is a schematic diagram of a satellite model of the present invention;

Fig. 2 is the flow chart of the method for measuring the position and attitude of the satellite in orbit of the present invention;

Fig. 3 is the flow chart of contour point parallelogram fitting of the present invention;

Fig. 4 is an effect diagram of the parallelogram fitting of the satellite solar panel of the present invention.

Detailed ways

The surface area of the solar panel is large, and the target is more obvious after being unfolded, so the present invention uses the solar panel as the visual detection object. The surface of a solar panel is generally composed of multiple groups of rectangular arrays. The size of a single rectangle is too small, and the distance between the rectangles is small, so it is difficult to ensure that it can be completely distinguished in the image. Therefore, a single solar panel as a whole is used as a visual inspection object. The edges of the small rectangles in the sailboard and the spaces between the rectangles will cause a lot of interference to image recognition. The on-orbit satellite pose measurement method based on parallelogram fitting proposed by the present invention can eliminate the interference inside the outline. Fitting realizes the detection of solar panels.

The purpose of the present invention is to use the solar panels on the satellites as detection objects in tasks such as satellite accompanying flight and satellite approaching, and use the mode of visual detection to detect the position and posture of the solar panels. There are many textures, which have a great influence on the extraction of the target edge. After obtaining the contour in the image through image processing, use the minimum rectangular envelope and other methods to process and combine the contour, and combine the pose of the solar panel in Cartesian space and in the image. The detection conditions are set in the position, and the contours corresponding to the three sailboards are extracted. Then, the four corner points of the contour are solved by using the contour point parallelogram fitting algorithm proposed by the present invention. The present invention has many applicable occasions and high fitting precision.

A new parallelogram fitting algorithm proposed by the present invention is used to fit objects with a rectangular surface shape in the image plane in Cartesian space. This algorithm uses the least square method to solve the best fit of all contour points. The best straight line, using it to preliminarily confirm the point set used to fit two parallel lines, and further confirm the point set used to fit the other two parallel lines, on this basis, delete the offset by iteration The points of the fitted line are solved for the four sides of the fitted parallelogram. The algorithm gradually approaches the best fitting line by iterative method, and can eliminate the interference of points with large errors, and is applicable to both closed contour points and discrete point sets.

The measurement object of the technical solution of the present invention is a satellite. As shown in Figure 1, there are three solar panels on the surface of the satellite, and these three solar panels are used as the detection objects of the present invention.

As shown in Figure 2, the present invention discloses a method for measuring the position and attitude of satellites in orbit based on parallelogram fitting, comprising the following steps:

Step A, collecting images; using a monocular or multi-camera to capture satellite images.

Step B, designing an image processing algorithm to extract the contours therein.

Step C, combining the contour points according to the set judgment conditions.

Step D, target contour selection.

Step E, using a parallelogram fitting algorithm to perform parallelogram fitting on the target contour points.

Step F, extract corner points and calculate the pose of the target satellite.

In said step B, comprising:

Step B1, image distortion removal.

In step B2, according to the area of the target satellite in the image, a ROI (local interest) search area is set to improve calculation efficiency.

Step B3, converting the image into an HLS format that is more suitable for machine judgment.

Step B4, performing median filtering on the image to reduce noise interference.

In step B5, the target area is extracted according to the HLS value of each pixel of the image, and the image is converted into a binary image.

Step B6, use the morphological closing operation to reduce the interference in the image, fill the gaps in the target area, and reduce the extraction error.

Step B7, extract the contour.

In said step C, also include:

Step C1, use the minimum enveloping rectangle to fit each contour point set, and obtain the center point, side length, rotation angle around the center point, etc. of the fitted rectangle.

Step C2, starting from the first contour, sequentially judge the relationship between the distance between the two contours and the side length of the contour fitting rectangle, if the difference is greater than the threshold, merge the contour point set into the previous contour point set, and delete the latter one Set of contour points.

Step C3, return to step C1, repeat steps C2 and C3 until no contours can be merged.

Step C4, filter the contour point set according to the number of points in each contour point set, and delete the contour with a small number of contour points.

In the step D, the target is extracted as the contour corresponding to the solar panel, specifically including:

Step D1, select the 3 contours with the largest number of contour points as the contours of the 3 target solar panels;

Step D2, according to the positional relationship of the three sailboards in the image plane, distinguish the outlines of the three sailboards. Among the center point coordinates (u, v), the sailboard with the smallest u is the leftmost sailboard, the one with u in the middle is the middle sailboard, and u The largest is the rightmost sailboard.

As shown in Figure 3, in said step E, also include:

Step E1, obtain the contour point set to be fitted, and calculate the center point of the contour, that is, the average value of the (u, v) coordinates of each point.

Step E2, using the least squares method to obtain an optimal fitting straight line ax+by+c=0, the straight line passing through the center point of the contour point set.

Step E3, determining point sets fitting two edge lines of a parallelogram approximately parallel to the optimal straight line.

Step E4, using the least squares method to fit the best fitting straight lines of the upper and lower contour point sets respectively, the two fitted straight lines must be parallel straight lines, because the two opposite sides of the fitted parallelogram should be parallel respectively, using two contour points After fitting two straight lines, the deflection angle of the straight line is calculated, and the weighted average of the deflection angle of the straight line is calculated according to the number of elements of the two contour point sets, and then a, b, c1, c2 of the two straight lines are determined.

Step E5, determining point sets for fitting the other two parallel lines of the parallelogram.

Step E6, fitting another two parallel straight lines according to the point set obtained in step E5.

Step E7, according to the preliminarily determined point set for fitting the four sides of the parallelogram and the preliminarily confirmed straight line equations of the four sides, gradually approach the optimal straight line equation of the four sides of the parallelogram in an iterative manner.

In said step E3, also include:

Step E30 , according to the positional relationship between the contour points and the straight line, the contour points are divided into two parts: points above the straight line (including on the straight line) and points below the straight line.

Step E31, calculate the distance from each contour point to the straight line, and find the maximum distances from the points above the straight line and the points below the straight line to the straight line respectively.

Step E32, for the points above and below the straight line, assign them to different point sets according to the difference between the distance from each point to the straight line and the maximum distance obtained in step E31.

Step E33, for the upper and lower points, if the number of elements in the farthest point set is the largest, or the number of elements in the farthest point set is greater than the threshold, then the farthest point set is the set where the contour edge points are located, otherwise the remaining point set has the largest number of elements The point set is the set where the contour edge points are located. Because in general, the contour points should be distributed in relatively concentrated areas, and the contour points should generally be at the edge of the entire contour point set.

In said step E5, also include:

Step E50, finding the point set in step E4 whose distance from the two fitted straight lines is less than the threshold and on the edge of the point set as the starting search point for the other two sides of the parallelogram to be fitted.

Step E51, solve the straight line passing through two points through the two pairs of initial search points obtained in step E50, use these two straight lines as the two initial sidelines of the parallelogram to be fitted, and search for points whose distance from the straight line is smaller than the threshold value as the fitted line. Combine the point sets of the two sidelines of the parallelogram to fit the straight line formed by the two part point sets, the method is the same as step E4.

In said step E7, also include:

Step E70, for the point set used to fit the four sides of the parallelogram, determine the distance between each point and the fitted straight line, if it is less than the threshold, keep it, and if it is greater than the threshold, delete it.

Step E71, using the point set processed in step E70 to fit two pairs of parallel edge lines of a parallelogram, the method is the same as step E4.

Step E72, judge whether the points deleted twice before and after are less than the threshold value, if yes, then end, if not, then return to step E70; that is, repeat step E70, step E71, continue to judge until the four points after the two times of processing The points deleted from the set are all smaller than the threshold, and the distance threshold used in step E70 will decrease as the number of iterations increases.

In said step F, also include:

Step F1, solving the four corner points formed by the intersection of the four fitted straight lines, that is, the four corner points of the fitted parallelogram, and the three contours have a total of 12 corner points.

Step F2, find the pose of the satellite in the camera coordinate system.

In the step F2, if the three-dimensional size of the satellite sailboard is known, the P12P algorithm can be used for the monocular camera. If the fitting error of a certain corner point is large, the corner point can be excluded and the remaining corner points can be used for fitting. For a binocular camera or a multi-eye camera, the P12P algorithm can be used to calculate the satellite pose for each monocular camera image and then take the average value. After obtaining the position of a corner point, the pose of the satellite is obtained.

As shown in Figure 4, it is the fitting effect of the three solar panels of the satellite. The connecting lines are the four sides of the parallelogram, the intersection points of each side, and the dots are the corner points of the fitted parallelogram. From the fitting image It can be seen that the fitting accuracy of the algorithm is very high, and the calculation amount of the whole algorithm is relatively small compared with other algorithms such as straight line extraction.

Compared with the prior art, the present invention has the following advantages:

1. The present invention provides a method for detecting satellites and measuring the position and attitude of satellites. The solar panels that are basically all equipped with satellites are used as the feature detection object. For the features that have many lines on the solar panels and have a great influence on the edge extraction of the target, through Image processing After obtaining the contours in the image, use methods such as the minimum rectangular envelope to process and combine the contours, and combine the pose of the solar panels in the Cartesian space and the position in the image to set the detection conditions, and extract the corresponding values of the three sails. contour, and then use the contour parallelogram fitting algorithm proposed by the present invention to solve the four corner points of the contour. The method for measuring the position and attitude of the satellite in orbit of the present invention is applicable to many occasions and has high fitting accuracy.

2. The present invention provides a new parallelogram fitting algorithm. There are many objects whose surface shape is rectangular in Cartesian space. Theoretically, its shape is a parallelogram in the image plane. This algorithm uses the least square method to solve the best straight line fitting of all contour points, and preliminarily confirms that it is used for fitting The point sets of two parallel lines are combined, and the point sets used to fit the other two parallel lines are further confirmed. For the four sides of a parallelogram, the algorithm gradually approximates the best fitting line through iterative methods, and can eliminate the interference of points with large errors. It is applicable to both closed contour points and discrete point sets.

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (10)

1. The on-orbit satellite pose measurement method based on parallelogram fitting is characterized by comprising the following steps of:

step A, collecting an image;

step B, designing an image processing algorithm, and extracting the outline in the image processing algorithm;

step C, combining contour points according to the set judging conditions;

step D, selecting a target contour;

e, performing parallelogram fitting on the target contour points by utilizing a parallelogram fitting algorithm;

and F, extracting corner points and resolving the pose of the target satellite.

2. The method for measuring the pose of an in-orbit satellite according to claim 1, further comprising, in the step B:

step B1, removing image distortion;

step B2, setting a region of interest (ROI) searching region according to the region of the target satellite in the image;

step B3, converting the image into an HLS format suitable for machine judgment;

step B4, median filtering is carried out on the image, so that noise interference is reduced;

step B5, extracting a target area according to HLS values of all pixel points of the picture, and converting the image into a binary image;

step B6, interference in the image is reduced by using morphological closing operation, gaps of the target area are filled, and extraction errors are reduced;

and B7, extracting the outline.

3. The method for measuring the pose of an in-orbit satellite according to claim 1, wherein in said step C, further comprising:

step C1, fitting each contour point set by using a minimum envelope rectangle to obtain a center point, side length and rotation angle around the center point of the rectangle after fitting the rectangle;

step C2, starting from the first contour, judging the relation between the distance between the two contours and the rectangular side length of the contour fitting in sequence, if the difference is greater than a threshold value, merging contour point sets into the previous contour point set, and deleting the latter contour point set;

step C3, returning to the step C1, and repeating the step C2 and the step C3 until no contour can be combined;

and C4, screening the contour point sets according to the number of the contour point sets, and deleting the contours with smaller contour point numbers.

4. The method for measuring the pose of an in-orbit satellite according to claim 1, wherein in the step D, the extraction target is a contour corresponding to a solar sailboard, and the method specifically comprises:

step D1, selecting the contour with the largest number of contour points as the contour of the target solar sailboard;

and D2, distinguishing the contour of the sailboard according to the position relation of the sailboard on the image plane, wherein in the central point coordinates (u, v), u is the leftmost sailboard, u is the middle sailboard in the middle, and u is the rightmost sailboard.

5. The method for measuring the pose of an in-orbit satellite according to claim 1, further comprising, in said step E:

e1, obtaining a contour point set to be fitted, and solving the average value of the central point of the contour, namely the coordinates of each point (u, v);

step E2, using a least square method to calculate a fitted optimal straight line ax+by+c=0, the straight line passing through the center point of the contour point set;

e3, determining a point set of two edge lines of a parallelogram which is approximately parallel to the optimal straight line;

e4, respectively fitting the best fit straight lines of the upper contour point set and the lower contour point set by using a least square method, wherein the two fitted straight lines are required to be parallel straight lines, respectively fitting the two straight lines by using the two contour point sets, then solving the deflection angle of the straight lines, solving the weighted average value of the deflection angle of the straight lines according to the element quantity of the two contour point sets, and further determining a, b, c1 and c2 of the two straight lines;

e5, determining a point set for fitting the other two parallel lines of the parallelogram;

e6, fitting two other parallel straight lines according to the point set obtained in the step E5;

and E7, gradually approaching the optimal linear equation of the four sides of the parallelogram in an iterative mode according to the preliminarily determined point set fitting the four sides of the parallelogram and the preliminarily determined linear equation of the four sides.

6. The method for measuring the pose of an in-orbit satellite according to claim 5, further comprising, in said step E3:

e30, dividing the contour point into a point above the straight line and a point below the straight line according to the position relation between the contour point and the straight line, wherein the point above the straight line comprises a point on the straight line;

e31, calculating the distance from each contour point to a straight line, and finding the maximum distance from the point above the straight line and the point below the straight line to the straight line respectively;

e32, for points above and below the straight line, respectively distributing the points to different point sets according to the difference between the distance from each point to the straight line and the maximum distance obtained in the step E31;

and E33, for the points above and below, if the number of elements of the furthest point set is the largest, or the number of elements of the furthest point set is larger than a threshold, the furthest point set is the set where the contour edge points are located, otherwise, the point set with the largest number of elements in the rest point sets is the set where the contour edge points are located.

7. The method for measuring the pose of an in-orbit satellite according to claim 5, wherein in said step E5, further comprising:

e50, solving the points in the point set in the step E4, which have the distance between the two straight lines to be fitted smaller than a threshold value and are at the edges of the point set, as initial search points of the other two sides of the parallelogram to be fitted;

and E51, solving a straight line passing through the two points through the two pairs of initial search points obtained in the step E50, taking the two straight lines as two initial edges of the parallelogram to be fitted, searching points with the distance of the straight line smaller than a threshold value as point sets for fitting the two edges of the parallelogram, and fitting the straight line formed by the two part point sets, wherein the method is the same as that of the step E4.

8. The method for measuring the pose of an in-orbit satellite according to claim 5, further comprising, in said step E7:

e70, judging the distance between each point and the straight line of the fitted parallelogram for the point set used for fitting the four edges of the parallelogram, if the distance is smaller than the threshold value, reserving the points, and if the distance is larger than the threshold value, deleting the points;

step E71, fitting two pairs of parallel edge lines of the parallelogram by using the point set processed in the step E70, wherein the method is the same as that in the step E4;

and E72, judging whether the two deleted points are smaller than a threshold value, if yes, ending, otherwise, returning to the step E70, wherein the distance threshold value used in the step E70 is reduced along with the increase of the iteration times.

9. The method for measuring the pose of an in-orbit satellite according to claim 1, further comprising, in said step F:

step F1, solving four corner points formed by intersecting the fitted four straight lines;

and F2, calculating the pose of the satellite in a camera coordinate system.

10. The method according to claim 9, wherein in the step F2, if the three-dimensional size of the solar array is known, the P12P algorithm is used for the monocular camera, if the fitting error of a certain corner point is large, the corner point can be eliminated, the remaining corner point fitting is used, the P12P algorithm is used for solving Wei Xingwei pose for each monocular camera image for the binocular camera or the multi-view camera, and then the average value is obtained, and if the three-dimensional size of the solar array is not known, the binocular camera or the multi-view camera is used for solving the position of each corner point to obtain the pose of the satellite.

Images