[go: up one dir, main page]

CN113790719B - A UAV inertial/visual landing navigation method based on line features - Google Patents

A UAV inertial/visual landing navigation method based on line features Download PDF

Info

Publication number
CN113790719B
CN113790719B CN202110928170.6A CN202110928170A CN113790719B CN 113790719 B CN113790719 B CN 113790719B CN 202110928170 A CN202110928170 A CN 202110928170A CN 113790719 B CN113790719 B CN 113790719B
Authority
CN
China
Prior art keywords
coordinate system
line
runway
axis
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110928170.6A
Other languages
Chinese (zh)
Other versions
CN113790719A (en
Inventor
尚克军
扈光锋
王大元
裴新凯
段昊雨
明丽
庄广琛
刘崇亮
王海军
焦浩
李茜茜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Automation Control Equipment Institute BACEI
Original Assignee
Beijing Automation Control Equipment Institute BACEI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Automation Control Equipment Institute BACEI filed Critical Beijing Automation Control Equipment Institute BACEI
Priority to CN202110928170.6A priority Critical patent/CN113790719B/en
Publication of CN113790719A publication Critical patent/CN113790719A/en
Application granted granted Critical
Publication of CN113790719B publication Critical patent/CN113790719B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1656Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an unmanned aerial vehicle inertial/visual landing navigation method based on line characteristics, which comprises the steps of firstly, collecting images of an airport runway, extracting real-time characteristics of the runway edge, and obtaining a linear equation of the edge and a central line; calculating Plucker coordinates of the boundary line equation through the width of the airport runway and the camera internal reference matrix which are bound in advance; calculating an equation of an infinite distance blanking point and a blanking line by two equidistant parallel lines, solving an attitude transfer matrix between a real-time world coordinate system and a camera coordinate system of the unmanned aerial vehicle by a simultaneous equation set, and solving the attitude, lateral and vertical positions; and taking the position information calculated by the visual landing system as an observed quantity, and fusing the observed quantity with a Kalman filter constructed by the navigation information output by inertial navigation to realize a continuous and autonomous navigation positioning function. The invention solves the problems of accumulated divergence of inertial navigation errors along with time and larger noise of visual navigation resolving results.

Description

一种基于线特征的无人机惯性/视觉着陆导航方法A UAV inertial/visual landing navigation method based on line features

技术领域Technical Field

本发明属于导航技术领域,具体涉及一种无人机着陆导航方法。The invention belongs to the field of navigation technology, and in particular relates to a drone landing navigation method.

背景技术Background Art

无人机视觉着陆的位姿解算过程中所应用的线特征为机场跑道两条边线、中心线以及远处的消隐线。与点特征不同,线特征不易受光照、飞行距离以及高度等因素的影响,具有更好的鲁棒性,但是由于消隐线的特征不够明显难以通过特征提取的方式识别,需要一种新的方案获取像平面内消隐线方程。同时需要考虑到视觉导航方式的位姿解算结果不够平滑且会随无人机快速机动发生跳变,需要一种滤波方法将惯性与视觉两种导航结果进行融合。The line features used in the posture solution of UAV visual landing are the two side lines of the airport runway, the center line, and the distant hidden line. Unlike point features, line features are not easily affected by factors such as lighting, flight distance, and altitude, and have better robustness. However, since the features of the hidden line are not obvious enough, it is difficult to identify them through feature extraction. A new solution is needed to obtain the equation of the hidden line in the image plane. At the same time, it is necessary to consider that the posture solution results of the visual navigation method are not smooth enough and will jump with the rapid maneuvering of the UAV. A filtering method is needed to fuse the inertial and visual navigation results.

发明内容Summary of the invention

本发明提供一种基于线特征的无人机惯性/视觉着陆导航方法,解决了惯性导航误差随时间累积发散与视觉导航解算结果噪声较大的问题。The present invention provides an unmanned aerial vehicle inertial/visual landing navigation method based on line features, which solves the problems of cumulative divergence of inertial navigation errors over time and large noise in visual navigation solution results.

一种基于线特征的无人机惯性/视觉着陆导航方法,包括如下步骤:A UAV inertial/visual landing navigation method based on line features comprises the following steps:

(1)机场跑道数据采集(1) Airport runway data collection

建立机场坐标系、视觉坐标系、世界坐标系、摄像机坐标系、图像坐标系;对机场跑道进行图像采集,并对跑道边线进行实时特征提取,获取边线及中线的直线方程;Establish the airport coordinate system, visual coordinate system, world coordinate system, camera coordinate system, and image coordinate system; collect images of the airport runway, perform real-time feature extraction on the runway sidelines, and obtain the linear equations of the sidelines and centerlines;

(2)Plücker坐标表示(2) Plücker coordinate representation

通过提前装订的机场跑道宽度与相机内参矩阵,计算边线方程的Plücker坐标;The Plücker coordinates of the edge equation are calculated by using the pre-bound airport runway width and the camera internal parameter matrix;

(3)视觉测量位姿解算(3) Visual measurement pose calculation

得到跑道边线的Plücker坐标后,由两条等距平行线计算无穷远处消隐点及消隐线的方程,通过联立方程组解算无人机实时的世界坐标系与相机坐标系之间的姿态转移矩阵Cwc,并进行姿态与侧向、垂向位置的求解;After obtaining the Plücker coordinates of the runway edge, the equations of the vanishing point and the vanishing line at infinity are calculated by two equidistant parallel lines. The attitude transfer matrix C w c between the real-time world coordinate system and the camera coordinate system of the UAV is solved by the simultaneous equations, and the attitude and lateral and vertical positions are solved.

(4)基于惯性/视觉融合的组合导航(4) Combined navigation based on inertial/visual fusion

将视觉着陆系统解算出的位置信息作为观测量,与惯性导航输出的导航信息构建卡尔曼滤波器进行融合,实现连续自主的导航定位功能。The position information calculated by the visual landing system is used as the observation quantity, and the Kalman filter is constructed to fuse it with the navigation information output by the inertial navigation to realize continuous and autonomous navigation and positioning functions.

进一步地,步骤(1)中建立机场坐标系、视觉坐标系、世界坐标系、摄像机坐标系、图像坐标系,包括;Furthermore, in step (1), the airport coordinate system, the visual coordinate system, the world coordinate system, the camera coordinate system, and the image coordinate system are established, including:

机场坐标系,记为a系;以跑道着陆端起始线与跑道中心线的交点为原点oa;轴沿跑道中心线,前向为正;ya轴垂直于跑道平面,向上为正;za轴与跑道起始线重合,右向为正;oaxayaza构成右手坐标系;机场坐标系下某点的坐标用(xa,ya,za)表示;The airport coordinate system is denoted as the a system; the intersection of the runway landing end start line and the runway center line is the origin oa ; the axle is along the runway center line, and the forward direction is positive; the ya axis is perpendicular to the runway plane, and the upward direction is positive; the z a axis coincides with the runway start line, and the right direction is positive; oa x a y a z a constitutes a right-handed coordinate system; the coordinates of a point in the airport coordinate system are expressed as (x a , y a , z a );

视觉坐标系,记为v系;以光学系统的像方主点为原点ov;xv轴平行于光轴,前向为正;yv轴平行于成像平面坐标系的横轴,向上为正;zv轴与xv轴和yv轴构成右手坐标系,右向为正;Visual coordinate system, denoted as v system; the image principal point of the optical system is taken as the origin o v ; the x v axis is parallel to the optical axis, and the forward direction is positive; the y v axis is parallel to the horizontal axis of the imaging plane coordinate system, and the upward direction is positive; the z v axis, the x v axis and the y v axis form a right-handed coordinate system, and the right direction is positive;

世界坐标系,记为w系;以跑道着陆端瞄准点起始线与跑道中心线的交点为原点ow;xw轴与跑道起始线重合,右向为正;yw轴垂直于跑道平面,向下为正;zw轴沿跑道中心线,前向为正;owxwywzw构成右手坐标系;世界坐标系下某点的坐标用(xw,yw,zw)表示;The world coordinate system is recorded as the w system; the intersection of the runway landing end aiming point starting line and the runway centerline is taken as the origin o w ; the x w axis coincides with the runway starting line, and the right direction is positive; the y w axis is perpendicular to the runway plane, and the downward direction is positive; the z w axis is along the runway centerline, and the forward direction is positive; o w x w y w z w constitutes a right-handed coordinate system; the coordinates of a point in the world coordinate system are expressed as (x w ,y w ,z w );

摄像机坐标系,记为c系;以光学系统的像方主点为原点oc;当正对光学系统观察时,xc轴平行于成像平面坐标系的水平轴,左向为正;yc轴平行于成像平面坐标系的垂直轴,向下为正;zc轴指向观察者,并与xc轴和yc轴构成右手坐标系;The camera coordinate system is denoted as the c system; the image principal point of the optical system is taken as the origin o c ; when observing the optical system directly, the x c axis is parallel to the horizontal axis of the imaging plane coordinate system, and the left direction is positive; the y c axis is parallel to the vertical axis of the imaging plane coordinate system, and the downward direction is positive; the z c axis points to the observer and forms a right-handed coordinate system with the x c axis and the y c axis;

图像坐标系,记为i系;在摄像机光敏面所在的平面内建立图像坐标系,以图像左上角为原点,沿图像水平方向向右为图像坐标系的xi轴,沿图像垂直方向向下为图像坐标系的yi轴,图像坐标系的单位是像素。The image coordinate system is denoted as the i-system; the image coordinate system is established in the plane where the camera's photosensitive surface is located, with the upper left corner of the image as the origin, the x i- axis of the image coordinate system extending to the right along the horizontal direction of the image, and the y i- axis of the image coordinate system extending downward along the vertical direction of the image. The unit of the image coordinate system is pixel.

进一步地,Plücker坐标表示具体包括:Furthermore, the Plücker coordinate representation specifically includes:

在像空间图像坐标系下直线方程可描述为:The equation of the straight line in the image space coordinate system can be described as:

axi+byi+c=0ax i +by i +c = 0

因此,直线可用三维向量表示:Therefore, a straight line can be represented by a three-dimensional vector:

l=[a,b,c]T l=[a,b,c] T

而在物空间世界坐标系中设两点A、B的三维坐标分别为它们的齐次坐标为:过这两点的直线可用一个4×4的反对称齐次矩阵L表示,该矩阵称之为Plücker矩阵:In the object space world coordinate system, the three-dimensional coordinates of two points A and B are Their homogeneous coordinates are: The line passing through these two points can be represented by a 4×4 antisymmetric homogeneous matrix L, which is called the Plücker matrix:

L=ABT-BAT L=AB T -BA T

另外,直线L可用其方向向量与矩m来表示,称之为Plücker坐标,记为:Alternatively, the line L can be expressed by its direction vector It is expressed by moment m, which is called Plücker coordinates and is recorded as:

其中,是直线的方向向量,矩m是该直线和原点确定平面的法向量,即 in, is the direction vector of the line, and the moment m is the normal vector of the plane determined by the line and the origin, that is,

由此可得Plücker矩阵与Plücker坐标间的关系为:From this, the relationship between the Plücker matrix and the Plücker coordinates is:

在摄像机映射T的作用下,用Plücker矩阵定义的直线L表示图像坐标系下对应直线的像l:Under the action of the camera mapping T, the line L defined by the Plücker matrix represents the image l of the corresponding line in the image coordinate system:

其中,K为相机内参矩阵:Among them, K is the camera intrinsic parameter matrix:

为世界坐标系到摄像机坐标系的姿态转移矩阵,为摄像机坐标系原点在世界坐标系中的位置向量;s2只是直线中各参数的共有系数,因此[l]×可简化为: is the attitude transfer matrix from the world coordinate system to the camera coordinate system, is the position vector of the origin of the camera coordinate system in the world coordinate system; s 2 is just the common coefficient of the parameters in the straight line, so [l] × can be simplified to:

进一步地,步骤(3)中计算无穷远处消隐点及消隐线的方程包括:Furthermore, the equations for calculating the vanishing point and the vanishing line at infinity in step (3) include:

在无人机着陆过程中,视觉着陆系统对跑道线特征进行提取,其中跑道左右边线及中心线为一组平行线,可用来进行无穷远处消隐点坐标和消隐线方程的计算,设物空间内跑道上三条等距平行线为L0w、L1w、L2w,其在像平面内成像为l0i、l1i、l2i,则像空间内消隐点坐标可由以下关系求解:During the landing process of the UAV, the visual landing system extracts the runway line features, where the left and right side lines and the center line of the runway are a set of parallel lines that can be used to calculate the vanishing point coordinates and vanishing line equations at infinity. Assume that the three equidistant parallel lines on the runway in the object space are L 0w , L 1w , and L 2w , and their images in the image plane are l 0i , l 1i , and l 2i , then the vanishing point coordinates in the image space can be solved by the following relationship:

消隐线方程为:The equation of the vanishing line is:

l∞i=[(l0i×l2i)T(l1i×l2i)]l1i+2[(l0i×l1i)T(l2i×l1i)]l2i l ∞i = [(l 0i ×l 2i ) T (l 1i ×l 2i )]l 1i +2[(l 0i ×l 1i ) T (l 2i ×l 1i )]l 2i

假设物空间中一点A的四维齐次坐标那么过点A且方向为可表示为:Assume that the four-dimensional homogeneous coordinates of a point A in object space are Then it passes through point A and the direction is It can be expressed as:

当参数λ由0变化到∞时,A点由有限点变化到无穷远点,该点在世界坐标系下的坐标为:When the parameter λ changes from 0 to ∞, point A changes from a finite point to an infinite point, and the coordinates of this point in the world coordinate system are:

根据图像共轭方程获得消隐点与无人机姿态转移矩阵的关系为:According to the image conjugate equation, the relationship between the vanishing point and the UAV attitude transfer matrix is:

进一步整理方程得:Further rearranging the equation yields:

假设像空间消隐直线l∞i上某点为x,其在物空间的反向投影为一条方向为的直线;由点x在直线上,可得:Assume that a point on the image space blanking line l∞i is x, and its reverse projection in the object space is a line with the direction A straight line; Since point x is on the straight line, we can get:

xTl∞i=0x T l ∞i =0

利用与平面的法向量nπ正交可得:use Orthogonal to the plane's normal vector nπ, we get:

利用物空间中方向为的直线投影为像空间的点x,得:Using the direction in object space The straight line projection of is the point x in the image space, and we get:

对上式进行转置变换得:Transpose the above formula to get:

与前文联立可得像空间消隐线方程:Combined with the previous text, we can get the image space vanishing line equation:

所述通过联立方程组解算无人机实时的世界坐标系与相机坐标系之间的姿态转移矩阵并进行姿态与侧向、垂向位置的求解具体包括:The attitude transfer matrix between the real-time world coordinate system and the camera coordinate system of the drone is solved by the simultaneous equations. And solve the attitude and lateral and vertical positions, including:

联立上述公式可得方程组如下:Combining the above formulas, we can get the following equations:

式中,为姿态转移矩阵:In the formula, is the attitude transfer matrix:

机场跑道场景下,是物空间跑道中心线的单位方向向量,nπ是物空间跑道平面的单位法向量,nπ=[0,1,0]T,同理可得 In the airport runway scenario, is the unit direction vector of the centerline of the runway in object space, is the unit normal vector of the track plane in object space, = [0,1,0] T , and similarly we can get

令K-1p=[g1,g2,g3]T,KTl=[h1,h2,h3]T,(K-1p)×(KTl)=[e1,e2,e3]T,同时姿态转移矩阵Cwc为反对称阵,每一行、列元素的平方和为1,由上式可解:Let K -1 p =[g 1 ,g 2 ,g 3 ] T , K T l =[h 1 ,h 2 ,h 3 ] T ,(K -1 p )×(K T l )=[e 1 ,e 2 ,e 3 ] T , and the attitude transfer matrix C w c is an antisymmetric matrix, and the sum of the squares of each row and column element is 1. The above formula can be solved:

由此解得三个姿态角:The three attitude angles are obtained from this:

利用线方程计算相对位置:Calculate relative position using line equation:

其中:in:

将跑道边线方程带入上式:Substitute the runway edge equation into the above formula:

同理由直线L2确定:Similarly, we can determine from the straight line L2 :

由上两式求解α0、α2Solve α 0 and α 2 from the above two equations:

最后解得无人机在世界坐标系下的垂向、侧向位置ty、txFinally, the vertical and lateral positions ty and tx of the UAV in the world coordinate system are solved:

进一步地,步骤(4)中,惯性/视觉组合导航系统的卡尔曼滤波模型连续状态方程如下:Furthermore, in step (4), the continuous state equation of the Kalman filter model of the inertial/visual integrated navigation system is as follows:

式中,F(t)为t时刻连续状态方程状态转移矩阵,为t时刻系统随机噪声向量;Where F(t) is the state transfer matrix of the continuous state equation at time t, is the system random noise vector at time t;

滤波状态量分别为北天东速度误差、维度误差、高度误差、经度误差、北天东向失准角误差、载体系XYZ方向陀螺漂移、载体系XYZ方向加速度计零位;The filtering state quantities are north sky east velocity error, latitude error, altitude error, longitude error, north sky east misalignment angle error, gyro drift of the carrier system in XYZ direction, and accelerometer zero position of the carrier system in XYZ direction;

系统状态转移矩阵System state transition matrix

其中:in:

观测方程定义如下:The observation equation is defined as follows:

为观测噪声阵; To observe the noise array;

组合导航系统的观测量为机场坐标系惯性导航输出的垂向侧向位置与视觉着陆系统导航结果的差值:The observation quantity of the integrated navigation system is the difference between the vertical lateral position output by the inertial navigation of the airport coordinate system and the navigation result of the visual landing system:

H(t)=[03×3 M3×3 03×9]H(t)=[0 3×3 M 3×3 0 3×9 ]

其中, in,

本发明提出了无穷远元素以及其Plücker坐标表示方法,解决了物空间中平行线在像平面上的交点坐标无法求解问题。其次由跑道左右边线及中线一组等距平行线求解无穷远处的消隐线方程,解决了无法通过特征提取方法对消隐线进行识别的问题。最后通过消隐线方程进行无人机位姿求解并与惯性融合,解决了惯性导航误差随时间累积发散与视觉导航解算结果噪声较大的问题。The present invention proposes an infinite element and its Plücker coordinate representation method, which solves the problem that the coordinates of the intersection of parallel lines in the object space on the image plane cannot be solved. Secondly, the equation of the vanishing line at infinity is solved by a set of equidistant parallel lines on the left and right side lines and the center line of the runway, which solves the problem that the vanishing line cannot be identified by the feature extraction method. Finally, the UAV posture is solved by the vanishing line equation and integrated with the inertial, which solves the problem of the cumulative divergence of inertial navigation errors over time and the large noise in the visual navigation solution results.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1坐标系示意图;Fig. 1 Schematic diagram of coordinate system;

图2一组平行线相交于消隐点示意图;Fig. 2 is a schematic diagram of a group of parallel lines intersecting at a vanishing point;

图3直线的Plücker坐标含义示意图。Fig. 3 Schematic diagram showing the meaning of the Plücker coordinates of a straight line.

具体实施方式DETAILED DESCRIPTION

下面结合附图对本发明作进一步详细说明。The present invention will be further described in detail below in conjunction with the accompanying drawings.

本发明针对卫星拒止条件下无人机的自主着陆导航问题,开展基于线特征的惯性/视觉着陆导航方法研究,首先提出了无穷远元素以及其Plücker坐标表示方法,解决了物空间中平行线在像平面上的交点坐标无法求解问题。其次由跑道左右边线及中线一组等距平行线求解无穷远处的消隐线方程,解决了无法通过特征提取方法对消隐线进行识别的问题。最后通过消隐线方程进行无人机位姿求解并与惯性融合,解决了惯性导航误差随时间累积发散与视觉导航解算结果噪声较大的问题。Aiming at the autonomous landing navigation problem of UAVs under satellite denial conditions, the present invention conducts research on inertial/visual landing navigation methods based on line features. First, an infinite element and its Plücker coordinate representation method are proposed, which solves the problem that the coordinates of the intersection of parallel lines in the object space on the image plane cannot be solved. Secondly, the equation of the vanishing line at infinity is solved by a group of equidistant parallel lines on the left and right side lines and the center line of the runway, which solves the problem that the invisible line cannot be identified by the feature extraction method. Finally, the UAV posture is solved by the invisible line equation and integrated with the inertial, which solves the problem of the cumulative divergence of inertial navigation errors over time and the large noise in the visual navigation solution results.

1.无穷远元素及Plücker表示法1. Elements at Infinity and Plücker Notation

(1)坐标系定义(1) Coordinate system definition

如图1所示,建立机场坐标系、视觉坐标系、世界坐标系、摄像机坐标系、图像坐标系。As shown in Figure 1, the airport coordinate system, visual coordinate system, world coordinate system, camera coordinate system, and image coordinate system are established.

其中机场坐标系(a系):以跑道着陆端起始线与跑道中心线的交点为原点oa;轴沿跑道中心线,前向为正;ya轴垂直于跑道平面,向上为正;za轴与跑道起始线重合,右向为正;oaxayaza构成右手坐标系;机场坐标系下某点的坐标用(xa,ya,za)表示。Among them, the airport coordinate system (a system) takes the intersection of the start line of the runway landing end and the center line of the runway as the origin oa ; the axis is along the runway center line, and the forward direction is positive; the ya axis is perpendicular to the runway plane, and the upward direction is positive; the z a axis coincides with the start line of the runway, and the right direction is positive; oa x a y a z a constitutes a right-handed coordinate system; the coordinates of a point in the airport coordinate system are expressed by (x a , y a , z a ).

视觉坐标系(v系):着陆视觉导航系统坐标系,简称视觉坐标系;以光学系统的像方主点为原点ov;xv轴平行于光轴,前向为正;yv轴平行于成像平面坐标系的横轴,向上为正;zv轴与xv轴和yv轴构成右手坐标系,右向为正。Visual coordinate system (V system): landing visual navigation system coordinate system, referred to as visual coordinate system; the image principal point of the optical system is taken as the origin O V ; the x V axis is parallel to the optical axis, and the forward direction is positive; the y V axis is parallel to the horizontal axis of the imaging plane coordinate system, and the upward direction is positive; the z V axis, the x V axis and the y V axis form a right-handed coordinate system, and the right direction is positive.

世界坐标系(w系):以跑道着陆端瞄准点起始线与跑道中心线的交点为原点ow;xw轴与跑道起始线重合,右向为正;yw轴垂直于跑道平面,向下为正;zw轴沿跑道中心线,前向为正;owxwywzw构成右手坐标系;世界坐标系下某点的坐标用(xw,yw,zw)表示。World coordinate system (W system): The intersection of the starting line of the aiming point at the landing end of the runway and the center line of the runway is the origin o w ; the x w axis coincides with the starting line of the runway, and the right direction is positive; the y w axis is perpendicular to the runway plane, and the downward direction is positive; the z w axis is along the center line of the runway, and the forward direction is positive; o w x w y w z w constitute a right-handed coordinate system; the coordinates of a point in the world coordinate system are expressed by (x w , y w , z w ).

摄像机坐标系(c系):以光学系统的像方主点为原点oc;当正对光学系统观察时,xc轴平行于成像平面坐标系的水平轴,左向为正;yc轴平行于成像平面坐标系的垂直轴,向下为正;zc轴指向观察者,并与xc轴和yc轴构成右手坐标系。Camera coordinate system (C system): The image principal point of the optical system is taken as the origin oc ; when observing the optical system directly, the xc axis is parallel to the horizontal axis of the imaging plane coordinate system, and the left direction is positive; the yc axis is parallel to the vertical axis of the imaging plane coordinate system, and the downward direction is positive; the zc axis points to the observer and forms a right-handed coordinate system with the xc axis and the yc axis.

图像坐标系(i系):在摄像机光敏面所在的平面内建立图像坐标系,它是一个二维平面坐标系,以图像左上角为原点,沿图像水平方向向右为图像坐标系的xi轴,沿图像垂直方向向下为图像坐标系的yi轴,图像坐标系的单位是像素。Image coordinate system (i system): The image coordinate system is established in the plane where the camera's photosensitive surface is located. It is a two-dimensional plane coordinate system with the upper left corner of the image as the origin. The x i axis of the image coordinate system is to the right along the horizontal direction of the image, and the y i axis of the image coordinate system is to the bottom along the vertical direction of the image. The unit of the image coordinate system is pixel.

(2)无穷远元素(2) Elements at Infinity

在物空间中,两条平行线永不相交,在欧式空间基础上通过引入无穷远元素构建射影空间,平面内一组平行线相交于无穷远处唯一一点,称之为消隐点(VanishingPoint)。如图2所示。该点在像平面上的位置只跟摄像机的姿态有关而与摄像机的位置无关。In object space, two parallel lines never intersect. Based on Euclidean space, a projective space is constructed by introducing infinite elements. A set of parallel lines in a plane intersect at a unique point at infinity, which is called the vanishing point. As shown in Figure 2, the position of this point on the image plane is only related to the camera's posture and has nothing to do with the camera's position.

消隐点代表对应平行线的方向,不平行直线的无穷远点不同,平面上所有的无穷远点构成一条直线,即消隐线(Vanishing Line)。消隐线是空间内一组平行面在无穷远处的唯一交线。The vanishing point represents the direction of the corresponding parallel line. Unlike the infinity points of non-parallel lines, all the infinity points on the plane form a straight line, the vanishing line. The vanishing line is the only intersection line of a set of parallel planes in space at infinity.

(3)Plücker表示法(3) Plücker notation

在像空间图像坐标系下直线方程可描述为:The equation of the straight line in the image space coordinate system can be described as:

axi+byi+c=0ax i +by i +c = 0

因此,直线可用三维向量表示:Therefore, a straight line can be represented by a three-dimensional vector:

l=[a,b,c]T l=[a,b,c] T

而在物空间世界坐标系中设两点A、B的三维坐标分别为(3×1矩阵),那么它们的齐次坐标为:过这两点的直线可用一个4×4的反对称齐次矩阵L表示,该矩阵称之为Plücker矩阵。In the object space world coordinate system, the three-dimensional coordinates of two points A and B are (3×1 matrix), then their homogeneous coordinates are: The straight line passing through these two points can be represented by a 4×4 antisymmetric homogeneous matrix L, which is called the Plücker matrix.

L=ABT-BAT L=AB T -BA T

另外,直线L可用其方向向量与矩m来表示,称之为Plücker坐标,记为:Alternatively, the line L can be expressed by its direction vector It is expressed by moment m, which is called Plücker coordinates and is recorded as:

其中,是直线的方向向量,矩m(可表征ΔABC的面积或者O到直线L的距离)是该直线和原点确定平面的法向量,即(如图3所示):in, is the direction vector of the line, and the moment m (which can represent the area of ΔABC or the distance from O to the line L) is the normal vector of the plane determined by the line and the origin, that is, (As shown in Figure 3):

由此可得Plücker矩阵与Plücker坐标间的关系为:From this, the relationship between the Plücker matrix and the Plücker coordinates is:

在摄像机映射T的作用下,用Plücker矩阵定义的直线L表示图像坐标系下对应直线的像l:Under the action of the camera mapping T, the line L defined by the Plücker matrix represents the image l of the corresponding line in the image coordinate system:

其中,K为相机内参矩阵:Among them, K is the camera intrinsic parameter matrix:

为世界坐标系到摄像机坐标系的姿态转移矩阵,为摄像机坐标系原点在世界坐标系中的位置向量。s2只是直线中各参数的共有系数,因此[l]×可简化为: is the attitude transfer matrix from the world coordinate system to the camera coordinate system, is the position vector of the origin of the camera coordinate system in the world coordinate system. s 2 is just the common coefficient of the parameters in the straight line, so [l] × can be simplified to:

2.视觉测量位姿解算2. Visual measurement pose calculation

(1)消隐点及消隐线成像方程(1) Vanishing point and vanishing line imaging equations

在无人机着陆过程中,视觉着陆系统对跑道线特征进行提取,其中跑道左右边线及中心线为一组平行线,可用来进行无穷远处消隐点坐标和消隐线方程的计算,设物空间内跑道上三条等距平行线为L0w、L1w、L2w,其在像平面内成像为l0i、l1i、l2i,则像空间内消隐点坐标可由以下关系求解:During the landing process of the drone, the visual landing system extracts the runway line features, where the left and right side lines and the center line of the runway are a set of parallel lines that can be used to calculate the vanishing point coordinates and vanishing line equations at infinity. Assume that the three equidistant parallel lines on the runway in the object space are L 0w , L 1w , and L 2w , and their images in the image plane are l 0i , l 1i , and l 2i , then the vanishing point coordinates in the image space can be solved by the following relationship:

消隐线方程为:The equation of the vanishing line is:

l∞i=[(l0i×l2i)T(l1i×l2i)]l1i+2[(l0i×l1i)T(l2i×l1i)]l2i l ∞i = [(l 0i ×l 2i ) T (l 1i ×l 2i )]l 1i +2[(l 0i ×l 1i ) T (l 2i ×l 1i )]l 2i

假设物空间中一点A的四维齐次坐标那么过点A且方向为Assume that the four-dimensional homogeneous coordinates of a point A in object space are Then it passes through point A and the direction is

(三维单位列向量),可表示为: (three-dimensional unit column vector), which can be expressed as:

当参数λ由0变化到∞时,A点由有限点变化到无穷远点,该点在世界坐标系下的坐标为:When the parameter λ changes from 0 to ∞, point A changes from a finite point to an infinite point, and the coordinates of this point in the world coordinate system are:

根据图像共轭方程获得消隐点与无人机姿态转移矩阵的关系为:According to the image conjugate equation, the relationship between the vanishing point and the UAV attitude transfer matrix is:

进一步整理方程得:Further rearranging the equation yields:

假设像空间消隐直线l∞i上某点为x,其在物空间的反向投影为一条方向为的直线。由点x在直线上,可得:Assume that a point on the image space blanking line l∞i is x, and its reverse projection in the object space is a line with the direction Since point x is on the straight line, we can get:

xTl∞i=0x T l ∞i =0

利用与平面的法向量nπ正交可得:use Orthogonal to the plane's normal vector nπ, we get:

利用物空间中方向为的直线投影为像空间的点x,得:Using the direction in object space The straight line projection of is the point x in the image space, and we get:

对上式进行转置变换得:Transpose the above formula to get:

与前文联立可得像空间消隐线方程:Combined with the previous text, we can get the image space vanishing line equation:

(2)无人机位姿解算(2) UAV posture calculation

联立前文中公式可得方程组如下:Combining the formulas in the previous article, we can get the following system of equations:

式中,为姿态转移矩阵:In the formula, is the attitude transfer matrix:

机场跑道场景下,是物空间跑道中心线的单位方向向量,nπ是物空间跑道平面的单位法向量,nπ=[0,1,0]T,同理可得 In the airport runway scenario, is the unit direction vector of the centerline of the runway in object space, is the unit normal vector of the track plane in object space, = [0,1,0] T , and similarly we can get

令K-1p=[g1,g2,g3]T,KTl=[h1,h2,h3]T,(K-1p)×(KTl)=[e1,e2,e3]T,同时姿态转移矩阵Cwc为反对称阵,每一行、列元素的平方和为1,由上式可解:Let K -1 p =[g 1 ,g 2 ,g 3 ] T , K T l =[h 1 ,h 2 ,h 3 ] T ,(K -1 p )×(K T l )=[e 1 ,e 2 ,e 3 ] T , and the attitude transfer matrix C w c is an antisymmetric matrix, and the sum of the squares of each row and column element is 1. The above formula can be solved:

由此解得三个姿态角:The three attitude angles are obtained from this:

利用线方程计算相对位置:Calculate relative position using line equation:

其中:in:

将跑道边线方程带入上式:Substitute the runway edge equation into the above formula:

同理由直线L2确定:Similarly, we can determine from the straight line L2 :

由上两式求解α0、α2Solve α 0 and α 2 from the above two equations:

最后解得无人机在世界坐标系下的垂向、侧向位置ty、txFinally, the vertical and lateral positions ty and tx of the UAV in the world coordinate system are solved:

3.惯性/视觉融合方法3. Inertial/visual fusion method

惯性/视觉组合导航系统的卡尔曼滤波模型连续状态方程如下:The continuous state equation of the Kalman filter model of the inertial/visual integrated navigation system is as follows:

式中,F(t)为t时刻连续状态方程状态转移矩阵,为t时刻系统随机噪声向量。Where F(t) is the state transfer matrix of the continuous state equation at time t, is the system random noise vector at time t.

滤波状态量分别为北天东速度误差(单位:m/s)、维度误差(单位:rad)、高度误差(单位:m)、经度误差(单位:rad)、北天东向失准角误差(单位:rad)、载体系XYZ方向陀螺漂移(单位:rad/s)、载体系XYZ方向加速度计零位(单位:m/s2)。The filtering state quantities are north sky east velocity error (unit: m/s), latitude error (unit: rad), altitude error (unit: m), longitude error (unit: rad), north sky east misalignment angle error (unit: rad), gyro drift of the carrier system in the XYZ direction (unit: rad/s), and accelerometer zero position of the carrier system in the XYZ direction (unit: m/ s2 ).

系统状态转移矩阵System state transition matrix

其中:in:

观测方程定义如下:The observation equation is defined as follows:

为观测噪声阵。 is the observed noise array.

组合导航系统的观测量为机场坐标系惯性导航输出的垂向侧向位置与视觉着陆系统导航结果的差值,:The observation quantity of the integrated navigation system is the difference between the vertical lateral position output by the inertial navigation of the airport coordinate system and the navigation result of the visual landing system:

H(t)=[03×3 M3×3 03×9]H(t)=[0 3×3 M 3×3 0 3×9 ]

其中, in,

综上,给出了无人机着落阶段利用跑道左右边线和中线三条等距平行线的惯性/视觉导航方法。In summary, an inertial/visual navigation method using three equidistant parallel lines, the left and right side lines and the center line of the runway, is proposed during the landing phase of the UAV.

本发明具体通过以下4个流程实现无人机自主着陆:The present invention specifically realizes autonomous landing of the drone through the following four processes:

(1)机场跑道数据采集:(1) Airport runway data collection:

通过安装在无人机机头的前视红外视觉导航系统对机场跑道进行图像采集,并对跑道边线进行实时特征提取,获取边线及中线的直线方程。The forward-looking infrared vision navigation system installed on the nose of the UAV is used to collect images of the airport runway, and the real-time feature extraction of the runway sideline is performed to obtain the straight line equations of the sideline and centerline.

(2)Plücker坐标表示:(2) Plücker coordinate representation:

通过提前装订的机场跑道宽度与相机内参矩阵,根据上述所推公式计算边线方程的Plücker坐标。The Plücker coordinates of the edge equation are calculated according to the formula derived above by using the pre-bound airport runway width and the camera intrinsic parameter matrix.

(3)视觉测量位姿解算:(3) Visual measurement pose solution:

得到跑道边线的Plücker坐标后,由两条等距平行线计算无穷远处消隐点及消隐线的方程,通过联立方程组解算无人机实时的世界坐标系与相机坐标系之间的姿态转移矩阵并进行姿态与侧向、垂向位置的求解。After obtaining the Plücker coordinates of the runway edge, the equations of the vanishing point and the vanishing line at infinity are calculated by two equidistant parallel lines, and the attitude transfer matrix between the real-time world coordinate system and the camera coordinate system of the drone is solved by the simultaneous equations. And solve the attitude and lateral and vertical positions.

(4)基于惯性/视觉融合的组合导航:(4) Combined navigation based on inertial/visual fusion:

将视觉着陆系统解算出的位置信息作为观测量,与惯性导航输出的导航信息构建卡尔曼滤波器进行融合,实现连续自主的导航定位功能。The position information calculated by the visual landing system is used as the observation quantity, and the Kalman filter is constructed to fuse it with the navigation information output by the inertial navigation to realize continuous and autonomous navigation and positioning functions.

Claims (5)

1.一种基于线特征的无人机惯性/视觉着陆导航方法,其特征在于,包括如下步骤:1. A UAV inertial/visual landing navigation method based on line features, characterized in that it includes the following steps: (1)机场跑道数据采集(1) Airport runway data collection 建立机场坐标系、视觉坐标系、世界坐标系、摄像机坐标系、图像坐标系;对机场跑道进行图像采集,并对跑道边线进行实时特征提取,获取边线及中线的直线方程;Establish the airport coordinate system, visual coordinate system, world coordinate system, camera coordinate system, and image coordinate system; collect images of the airport runway, and perform real-time feature extraction on the runway sidelines to obtain the linear equations of the sidelines and centerlines; (2)Plücker坐标表示(2) Plücker coordinate representation 通过提前装订的机场跑道宽度与相机内参矩阵,计算边线方程的Plücker坐标;The Plücker coordinates of the edge equation are calculated by using the pre-bound airport runway width and the camera internal parameter matrix; (3)视觉测量位姿解算(3) Visual measurement pose calculation 得到跑道边线的Plücker坐标后,由两条等距平行线计算无穷远处消隐点及消隐线的方程,通过联立方程组解算无人机实时的世界坐标系与相机坐标系之间的姿态转移矩阵并进行姿态与侧向、垂向位置的求解;After obtaining the Plücker coordinates of the runway edge, the equations of the vanishing point and the vanishing line at infinity are calculated by two equidistant parallel lines, and the attitude transfer matrix between the real-time world coordinate system and the camera coordinate system of the drone is solved by the simultaneous equations. And solve the attitude and lateral and vertical positions; (4)基于惯性/视觉融合的组合导航(4) Combined navigation based on inertial/visual fusion 将视觉着陆系统解算出的位置信息作为观测量,与惯性导航输出的导航信息构建卡尔曼滤波器进行融合,实现连续自主的导航定位功能;所述观测量为机场坐标系惯性导航输出的垂向侧向位置与视觉着陆系统导航结果的差值。The position information calculated by the visual landing system is used as the observation quantity, and the Kalman filter is constructed and fused with the navigation information output by the inertial navigation to realize the continuous autonomous navigation and positioning function; the observation quantity is the difference between the vertical lateral position output by the inertial navigation of the airport coordinate system and the navigation result of the visual landing system. 2.根据权利要求1所述的一种基于线特征的无人机惯性/视觉着陆导航方法,其特征在于,步骤(1)中建立机场坐标系、视觉坐标系、世界坐标系、摄像机坐标系、图像坐标系,包括;2. The UAV inertial/visual landing navigation method based on line features according to claim 1 is characterized in that the airport coordinate system, visual coordinate system, world coordinate system, camera coordinate system and image coordinate system are established in step (1), including: 机场坐标系,记为a系;以跑道着陆端起始线与跑道中心线的交点为原点oa;轴沿跑道中心线,前向为正;ya轴垂直于跑道平面,向上为正;za轴与跑道起始线重合,右向为正;oaxayaza构成右手坐标系;机场坐标系下某点的坐标用(xa,ya,za)表示;The airport coordinate system is denoted as the a system; the intersection of the runway landing end start line and the runway center line is the origin oa ; the axle is along the runway center line, and the forward direction is positive; the ya axis is perpendicular to the runway plane, and the upward direction is positive; the z a axis coincides with the runway start line, and the right direction is positive; oa x a y a z a constitutes a right-handed coordinate system; the coordinates of a point in the airport coordinate system are expressed as (x a , y a , z a ); 视觉坐标系,记为v系;以光学系统的像方主点为原点ov;xv轴平行于光轴,前向为正;yv轴平行于成像平面坐标系的横轴,向上为正;zv轴与xv轴和yv轴构成右手坐标系,右向为正;Visual coordinate system, denoted as v system; the image principal point of the optical system is taken as the origin o v ; the x v axis is parallel to the optical axis, and the forward direction is positive; the y v axis is parallel to the horizontal axis of the imaging plane coordinate system, and the upward direction is positive; the z v axis, the x v axis and the y v axis form a right-handed coordinate system, and the right direction is positive; 世界坐标系,记为w系;以跑道着陆端瞄准点起始线与跑道中心线的交点为原点ow;xw轴与跑道起始线重合,右向为正;yw轴垂直于跑道平面,向下为正;zw轴沿跑道中心线,前向为正;owxwywzw构成右手坐标系;世界坐标系下某点的坐标用(xw,yw,zw)表示;The world coordinate system is recorded as the w system; the intersection of the runway landing end aiming point starting line and the runway centerline is taken as the origin o w ; the x w axis coincides with the runway starting line, and the right direction is positive; the y w axis is perpendicular to the runway plane, and the downward direction is positive; the z w axis is along the runway centerline, and the forward direction is positive; o w x w y w z w constitutes a right-handed coordinate system; the coordinates of a point in the world coordinate system are expressed as (x w ,y w ,z w ); 摄像机坐标系,记为c系;以光学系统的像方主点为原点oc;当正对光学系统观察时,xc轴平行于成像平面坐标系的水平轴,左向为正;yc轴平行于成像平面坐标系的垂直轴,向下为正;zc轴指向观察者,并与xc轴和yc轴构成右手坐标系;The camera coordinate system is denoted as the c system; the image principal point of the optical system is taken as the origin o c ; when observing the optical system directly, the x c axis is parallel to the horizontal axis of the imaging plane coordinate system, and the left direction is positive; the y c axis is parallel to the vertical axis of the imaging plane coordinate system, and the downward direction is positive; the z c axis points to the observer and forms a right-handed coordinate system with the x c axis and the y c axis; 图像坐标系,记为i系;在摄像机光敏面所在的平面内建立图像坐标系,以图像左上角为原点,沿图像水平方向向右为图像坐标系的xi轴,沿图像垂直方向向下为图像坐标系的yi轴,图像坐标系的单位是像素。The image coordinate system is denoted as the i-system; the image coordinate system is established in the plane where the camera's photosensitive surface is located, with the upper left corner of the image as the origin, the x i- axis of the image coordinate system extending to the right along the horizontal direction of the image, and the y i- axis of the image coordinate system extending downward along the vertical direction of the image. The unit of the image coordinate system is pixel. 3.根据权利要求2所述的一种基于线特征的无人机惯性/视觉着陆导航方法,其特征在于,Plücker坐标表示具体包括:3. The UAV inertial/visual landing navigation method based on line features according to claim 2, characterized in that the Plücker coordinate representation specifically includes: 在像空间图像坐标系下直线方程可描述为:The equation of the straight line in the image space coordinate system can be described as: axi+byi+c=0ax i +by i +c = 0 因此,直线可用三维向量表示:Therefore, a straight line can be represented by a three-dimensional vector: l=[a,b,c]T l=[a,b,c] T 而在物空间世界坐标系中设两点A、B的三维坐标分别为它们的齐次坐标为:过这两点的直线可用一个4×4的反对称齐次矩阵L表示,该矩阵称之为Plücker矩阵:In the object space world coordinate system, the three-dimensional coordinates of two points A and B are Their homogeneous coordinates are: The line passing through these two points can be represented by a 4×4 antisymmetric homogeneous matrix L, which is called the Plücker matrix: L=ABT-BAT L=AB T -BA T 另外,直线L可用其方向向量与矩m来表示,称之为Plücker坐标,记为:Alternatively, the line L can be expressed by its direction vector It is expressed by moment m, which is called Plücker coordinates and is recorded as: 其中,是直线的方向向量,矩m是该直线和原点确定平面的法向量,即 in, is the direction vector of the line, and the moment m is the normal vector of the plane determined by the line and the origin, that is, 由此可得Plücker矩阵与Plücker坐标间的关系为:From this, the relationship between the Plücker matrix and the Plücker coordinates is: 在摄像机映射T的作用下,用Plücker矩阵定义的直线L表示图像坐标系下对应直线的像l:Under the action of the camera mapping T, the line L defined by the Plücker matrix represents the image l of the corresponding line in the image coordinate system: 其中,K为相机内参矩阵:Among them, K is the camera intrinsic parameter matrix: 为世界坐标系到摄像机坐标系的姿态转移矩阵,为摄像机坐标系原点在世界坐标系中的位置向量;s2只是直线中各参数的共有系数,因此[l]×可简化为: is the attitude transfer matrix from the world coordinate system to the camera coordinate system, is the position vector of the origin of the camera coordinate system in the world coordinate system; s 2 is just the common coefficient of the parameters in the straight line, so [l] × can be simplified to: 4.根据权利要求3所述的一种基于线特征的无人机惯性/视觉着陆导航方法,其特征在于,步骤(3)中计算无穷远处消隐点及消隐线的方程包括:4. The UAV inertial/visual landing navigation method based on line features according to claim 3 is characterized in that the equations for calculating the vanishing point and the vanishing line at infinity in step (3) include: 在无人机着陆过程中,视觉着陆系统对跑道线特征进行提取,其中跑道左右边线及中心线为一组平行线,可用来进行无穷远处消隐点坐标和消隐线方程的计算,设物空间内跑道上三条等距平行线为L0w、L1w、L2w,其在像平面内成像为l0i、l1i、l2i,则像空间内消隐点坐标可由以下关系求解:During the landing process of the UAV, the visual landing system extracts the runway line features, where the left and right side lines and the center line of the runway are a set of parallel lines that can be used to calculate the vanishing point coordinates and vanishing line equations at infinity. Assume that the three equidistant parallel lines on the runway in the object space are L 0w , L 1w , and L 2w , and their images in the image plane are l 0i , l 1i , and l 2i , then the vanishing point coordinates in the image space can be solved by the following relationship: 消隐线方程为:The equation of the vanishing line is: l∞i=[(l0i×l2i)T(l1i×l2i)]l1i+2[(l0i×l1i)T(l2i×l1i)]l2i l ∞i = [(l 0i ×l 2i ) T (l 1i ×l 2i )]l 1i +2[(l 0i ×l 1i ) T (l 2i ×l 1i )]l 2i 假设物空间中一点A的四维齐次坐标那么过点A且方向为可表示为:Assume that the four-dimensional homogeneous coordinates of a point A in object space are Then it passes through point A and the direction is It can be expressed as: 当参数λ由0变化到∞时,A点由有限点变化到无穷远点,该点在世界坐标系下的坐标为:When the parameter λ changes from 0 to ∞, point A changes from a finite point to an infinite point, and the coordinates of this point in the world coordinate system are: 根据图像共轭方程获得消隐点与无人机姿态转移矩阵的关系为:According to the image conjugate equation, the relationship between the vanishing point and the UAV attitude transfer matrix is: 进一步整理方程得:Further rearranging the equation yields: 假设像空间消隐直线l∞i上某点为x,其在物空间的反向投影为一条方向为的直线;由点x在直线上,可得:Assume that a point on the image space blanking line l∞i is x, and its reverse projection in the object space is a line with the direction A straight line; Since point x is on the straight line, we can get: xTl∞i=0x T l ∞i =0 利用与平面的法向量nπ正交可得:use Orthogonal to the plane's normal vector nπ, we get: 利用物空间中方向为的直线投影为像空间的点x,得:Using the direction in object space The straight line projection of is the point x in the image space, and we get: 对上式进行转置变换得:Transpose the above formula to get: 与前文联立可得像空间消隐线方程:Combined with the previous text, we can get the image space vanishing line equation: 所述通过联立方程组解算无人机实时的世界坐标系与相机坐标系之间的姿态转移矩阵并进行姿态与侧向、垂向位置的求解具体包括:The attitude transfer matrix between the real-time world coordinate system and the camera coordinate system of the drone is solved by the simultaneous equations. And solve the attitude and lateral and vertical positions, including: 联立上述公式可得方程组如下:Combining the above formulas, we can get the following equations: 式中,为姿态转移矩阵:In the formula, is the attitude transfer matrix: 机场跑道场景下,是物空间跑道中心线的单位方向向量,nπ是物空间跑道平面的单位法向量,nπ=[0,1,0]T,同理可得 In the airport runway scenario, is the unit direction vector of the centerline of the runway in object space, is the unit normal vector of the track plane in object space, = [0,1,0] T , and similarly we can get 令K-1p=[g1,g2,g3]T,KTl=[h1,h2,h3]T,(K-1p)×(KTl)=[e1,e2,e3]T,同时姿态转移矩阵为反对称阵,每一行、列元素的平方和为1,由上式可解:Let K -1 p = [g 1 , g 2 , g 3 ] T , K T l = [h 1 , h 2 , h 3 ] T , (K -1 p ) × (K T l ) = [e 1 , e 2 , e 3 ] T , and the attitude transfer matrix It is an antisymmetric matrix, and the sum of the squares of each row and column element is 1. The above formula can be solved: 由此解得三个姿态角:The three attitude angles are obtained from this: 利用线方程计算相对位置:Calculate relative position using line equation: 其中:in: 将跑道边线方程带入上式:Substitute the runway edge equation into the above formula: 同理由直线L2确定:Similarly, we can determine from the straight line L2 : 由上两式求解α0、α2Solve α 0 and α 2 from the above two equations: 最后解得无人机在世界坐标系下的垂向、侧向位置ty、txFinally, the vertical and lateral positions ty and tx of the UAV in the world coordinate system are solved: 5.根据权利要求4所述的一种基于线特征的无人机惯性/视觉着陆导航方法,其特征在于,步骤(4)中,惯性/视觉组合导航系统的卡尔曼滤波模型连续状态方程如下:5. The method for unmanned aerial vehicle inertial/visual landing navigation based on line features according to claim 4 is characterized in that, in step (4), the continuous state equation of the Kalman filter model of the inertial/visual integrated navigation system is as follows: 式中,F(t)为t时刻连续状态方程状态转移矩阵,为t时刻系统随机噪声向量;Where F(t) is the state transfer matrix of the continuous state equation at time t, is the system random noise vector at time t; 滤波状态量分别为北天东速度误差、维度误差、高度误差、经度误差、北天东向失准角误差、载体系XYZ方向陀螺漂移、载体系XYZ方向加速度计零位;The filtering state quantities are north sky east velocity error, latitude error, altitude error, longitude error, north sky east misalignment angle error, gyro drift of the carrier system in XYZ direction, and accelerometer zero position of the carrier system in XYZ direction; 系统状态转移矩阵System state transition matrix 其中:in: 观测方程定义如下:The observation equation is defined as follows: 为观测噪声阵; To observe the noise array; 组合导航系统的观测量为机场坐标系惯性导航输出的垂向侧向位置与视觉着陆系统导航结果的差值:The observation quantity of the integrated navigation system is the difference between the vertical lateral position output by the inertial navigation of the airport coordinate system and the navigation result of the visual landing system: H(t)=[03×3M3×3 03×9]H(t)=[0 3×3 M 3×3 0 3×9 ] 其中,in, .
CN202110928170.6A 2021-08-13 2021-08-13 A UAV inertial/visual landing navigation method based on line features Active CN113790719B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110928170.6A CN113790719B (en) 2021-08-13 2021-08-13 A UAV inertial/visual landing navigation method based on line features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110928170.6A CN113790719B (en) 2021-08-13 2021-08-13 A UAV inertial/visual landing navigation method based on line features

Publications (2)

Publication Number Publication Date
CN113790719A CN113790719A (en) 2021-12-14
CN113790719B true CN113790719B (en) 2023-09-12

Family

ID=79181750

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110928170.6A Active CN113790719B (en) 2021-08-13 2021-08-13 A UAV inertial/visual landing navigation method based on line features

Country Status (1)

Country Link
CN (1) CN113790719B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114719849B (en) * 2022-02-07 2025-01-28 北京空间飞行器总体设计部 A method for optimizing feature selection of autonomous navigation sequence images during planetary soft landing
CN115690205B (en) * 2022-10-09 2023-12-05 北京自动化控制设备研究所 Visual relative pose measurement error estimation method based on point-line comprehensive characteristics
CN116380057B (en) * 2023-06-05 2023-08-29 四川腾盾科技有限公司 Unmanned aerial vehicle autonomous landing positioning method under GNSS refusing environment
CN117115598B (en) * 2023-08-17 2024-08-06 北京自动化控制设备研究所 Visual line feature extraction precision evaluation method
CN118293915B (en) * 2024-04-10 2025-04-01 北京自动化控制设备研究所 A filtering method for full-profile inertial and visual integrated navigation
CN119197577B (en) * 2024-11-22 2025-01-28 四川国软科技集团有限公司 A navigation method based on road edge

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3546116A1 (en) * 1985-12-24 1987-06-25 Heinz Mueller Method for displaying required and actual data on the flight conditions before and during landing, for a pilot
CN104517291A (en) * 2014-12-15 2015-04-15 大连理工大学 Pose measuring method based on coaxial circle characteristics of target
CN109341700A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 Fixed wing aircraft vision assists landing navigation method under a kind of low visibility
CN109544696A (en) * 2018-12-04 2019-03-29 中国航空工业集团公司西安航空计算技术研究所 A kind of airborne enhancing Synthetic vision actual situation Image Precision Registration of view-based access control model inertia combination
CN110634162A (en) * 2019-04-01 2019-12-31 青岛鑫慧铭视觉科技有限公司 Calibration method of structured light vision sensor based on concentric circles
CN111275764A (en) * 2020-02-12 2020-06-12 南开大学 Depth camera visual mileage measurement method based on line segment shadow

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3546116A1 (en) * 1985-12-24 1987-06-25 Heinz Mueller Method for displaying required and actual data on the flight conditions before and during landing, for a pilot
CN104517291A (en) * 2014-12-15 2015-04-15 大连理工大学 Pose measuring method based on coaxial circle characteristics of target
CN109341700A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 Fixed wing aircraft vision assists landing navigation method under a kind of low visibility
CN109544696A (en) * 2018-12-04 2019-03-29 中国航空工业集团公司西安航空计算技术研究所 A kind of airborne enhancing Synthetic vision actual situation Image Precision Registration of view-based access control model inertia combination
CN110634162A (en) * 2019-04-01 2019-12-31 青岛鑫慧铭视觉科技有限公司 Calibration method of structured light vision sensor based on concentric circles
CN111275764A (en) * 2020-02-12 2020-06-12 南开大学 Depth camera visual mileage measurement method based on line segment shadow

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
嵇盛育 ; 徐贵力 ; 冯文玲 ; 刘小霞 ; .基于红外视觉的无人机自主着舰合作目标的研究.红外技术.2007,(第10期),593-597. *

Also Published As

Publication number Publication date
CN113790719A (en) 2021-12-14

Similar Documents

Publication Publication Date Title
CN113790719B (en) A UAV inertial/visual landing navigation method based on line features
CN101916452B (en) Method for automatically stitching unmanned aerial vehicle remote sensing images based on flight control information
CN105865454B (en) A kind of Navigation of Pilotless Aircraft method generated based on real-time online map
CN101532841B (en) Aircraft Navigation and Positioning Method Based on Landmark Acquisition and Tracking
CN106155086B (en) A kind of Road Detection unmanned plane and its automatic cruising method
CN104618689B (en) Coastal waters spilled oil monitoring method and system based on unmanned plane
US10970924B2 (en) Reconstruction of a scene from a moving camera
CN106989744A (en) A kind of rotor wing unmanned aerial vehicle autonomic positioning method for merging onboard multi-sensor
CN110186468B (en) High-precision map making method and device for automatic driving
CN107194989A (en) The scene of a traffic accident three-dimensional reconstruction system and method taken photo by plane based on unmanned plane aircraft
CN106017463A (en) Aircraft positioning method based on positioning and sensing device
JP2008186145A (en) Aerial image processing apparatus and aerial image processing method
CN112710311B (en) A terrain-adaptive UAV 3D real scene reconstruction aerial point automatic planning method
CN109562844A (en) The assessment of automatic Landing topographical surface and relevant system and method
KR102239562B1 (en) Fusion system between airborne and terrestrial observation data
Haala et al. Dense multiple stereo matching of highly overlapping UAV imagery
CN114066972B (en) A UAV autonomous positioning method based on monocular vision
CN115371673A (en) A binocular camera target location method based on Bundle Adjustment in an unknown environment
Bazin et al. UAV attitude estimation by vanishing points in catadioptric images
CN116957360A (en) A UAV-based space observation and reconstruction method and system
CN116989772A (en) Air-ground multi-mode multi-agent cooperative positioning and mapping method
CN116168171A (en) Real-time Dense Reconstruction Method for Swarm UAVs
CN110223233A (en) A kind of unmanned plane based on image mosaic builds drawing method
Majdik et al. Micro air vehicle localization and position tracking from textured 3d cadastral models
CN113155126A (en) Multi-machine cooperative target high-precision positioning system and method based on visual navigation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant