Papers by Daniele Mortari
The Journal of the astronautical sciences, Feb 1, 2024
This article introduces physics-informed neural networks (PINNs) to the field of
motion plannin... more This article introduces physics-informed neural networks (PINNs) to the field of
motion planning by utilizing a PINN framework as the steering function in the kino-
dynamic rapidly-exploring random tree (RRT*) algorithm. The goal of this paper
is to show that PINN-based methods can be used successfully for aerospace motion
planning applications. We test the RRT* algorithm coupled with PINN steering,
what we call PINN-RRT*, by solving spacecraft energy-optimal motion planning
problems governed by the Hill–Clohessy–Wiltshire (HCW) equations of motion and
nonlinear equations of relative motion (NERM), where a deputy satellite must ren-
rendezvous with a chief satellite while avoiding spherical keep-out-zones and complying with an approach corridor. The particular PINN framework we employ approximates the solution of nonlinear two-point boundary value problems (TPBVPs),
which must be solved to form connections between waypoints in the RRT* tree, via
the Theory of Functional Connections (TFC). TFC enables the PINN to analytically
satisfy the boundary conditions (BCs) of the TPBVP. Thus, the admissible solution
search space of each nonlinear TPBVP is reduced to just the trajectories that already
satisfy the BCs. Using our proposed approach, each energy-optimal TPBVP solution
during the run-time of the PINN-RRT* algorithm was computed in centiseconds
and with an average error on the order of machine epsilon for both the HCW and
NERM dynamics.
Differential equations (DEs) are used as numerical models to describe physical phenomena througho... more Differential equations (DEs) are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. While there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Functional Connections (TFC) with one based on least-squares support vector machines (LS-SVM). The TFC method uses a constrained expression, an expression that always satisfies the DE constraints, which transforms the process of solving a DE into solving an unconstrained optimization problem that is ultimately solved via least-squares (LS). In addition to individual analysis, the two methods are merged into a new methodology, called constrained SVMs (CSVM), by incorporating the LS-SVM method into the TFC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: One first order linear ordinary differential equation (ODE), one first order nonlinear ODE, one second order linear ODE, and one two-dimensional linear partial differential equation (PDE). Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean squared error on the training and test sets. In general, TFC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) than the LS-SVM and CSVM approaches.
This article presents a reformulation of the Theory of Functional Connections: a general methodol... more This article presents a reformulation of the Theory of Functional Connections: a general methodology for functional interpolation that can embed a set of user-specified linear constraints. The reformulation presented in this paper exploits the underlying functional structure presented in the seminal paper on the Theory of Functional Connections to ease the derivation of these interpolating functionals-called constrained expressions-and provides rigorous terminology that lends itself to straightforward derivations of mathematical proofs regarding the properties of these constrained expressions. Furthermore, the extension of the technique to and proofs in n-dimensions is immediate through a recursive application of the univariate formulation. In all, the results of this reformulation are compared to prior work to highlight the novelty and mathematical convenience of using this approach. Finally, the methodology presented in this paper is applied to two partial differential equations with different boundary conditions, and, when data is available, the results are compared to state-of-the-art methods.
This study shows how to obtain least-squares solutions to initial and boundary value problems of ... more This study shows how to obtain least-squares solutions to initial and boundary value problems of ordinary nonlinear differential equations. The proposed method begins using an approximate solution obtained by any existing integrator. Then, a least-squares fitting of this approximate solution is obtained using a constrained expression, derived from Theory of Connections. In this expression, the differential equation constraints are embedded and are always satisfied. The resulting constrained expression is then used as an initial guess in a Newton iterative process that increases the solution accuracy to machine error level in no more than two iterations for most of the problems considered. An analysis of speed and accuracy has been conducted for this method using two nonlinear differential equations. For non-smooth solutions or for long integration times, a piecewise approach is proposed. The highly accurate value estimated at the final time is then used as the new initial guess for the next time range, and this process is repeated for subsequent time ranges. This approach has been applied and validated solving the Duffing oscillator obtaining a final solution error on the order of 10 -12 . To complete the study, a final numerical test is provided for a boundary value problem with a known solution.
This paper extends the univariate Theory of Connections, introduced in (Mortari, 2017), to the mu... more This paper extends the univariate Theory of Connections, introduced in (Mortari, 2017), to the multivariate case on rectangular domains with detailed attention to the bivariate case. In particular, it generalizes the bivariate Coons surface, introduced by (Coons, 1984), by providing analytical expressions, called constrained expressions, representing all possible surfaces with assigned boundary constraints in terms of functions and arbitrary-order derivatives. In two dimensions, these expressions, which contain a freely chosen function, g(x, y), satisfy all constraints no matter what the g(x, y) is. The boundary constraints considered in this article are Dirichlet, Neumann, and any combinations of them. Although the focus of this article is on two-dimensional spaces, the final section introduces the Multivariate Theory of Connections, validated by mathematical proof. This represents the multivariate extension of the Theory of Connections subject to arbitrary-order derivative constraints in rectangular domains. The main task of this paper is to provide an analytical procedure to obtain constrained expressions in any space that can be used to transform constrained problems into unconstrained problems. This theory is proposed mainly to better solve PDE and stochastic differential equations.
This study shows how to obtain least-squares solutions to initial value problems (IVPs), boundary... more This study shows how to obtain least-squares solutions to initial value problems (IVPs), boundary value problems (BVPs), and multi-value problems (MVPs) for nonhomogeneous linear differential equations (DEs) with nonconstant coefficients of any order. However, without loss of generality, the approach has been applied to second-order DEs. The proposed method has two steps. The first step consists of writing a constrained expression, that has the DE constraints embedded. These kind of expressions are given in terms of a new unknown function, g(t), and they satisfy the constraints, no matter what g(t) is. The second step consists of expressing g(t) as a linear combination of m independent known basis functions. Specifically, orthogonal polynomials are adopted for the basis functions. This choice requires rewriting the DE and the constraints in terms of a new independent variable, x ∈ [-1, +1]. The procedure leads to a set of linear equations in terms of the unknown coefficients of the basis functions that are then computed by least-squares. Numerical examples are provided to quantify the solutions' accuracy for IVPs, BVPs and MVPs. In all the examples provided, the least-squares solution is obtained with machine error accuracy.
This article presents a new methodology called Deep Theory of Functional Connections (TFC) that e... more This article presents a new methodology called Deep Theory of Functional Connections (TFC) that estimates the solutions of partial differential equations (PDEs) by combining neural networks with the TFC. The TFC is used to transform PDEs into unconstrained optimization problems by analytically embedding the PDE's constraints into a "constrained expression" containing a free function. In this research , the free function is chosen to be a neural network, which is used to solve the now unconstrained optimization problem. This optimization problem consists of minimizing a loss function that is chosen to be the square of the residuals of the PDE. The neural network is trained in an unsupervised manner to minimize this loss function. This methodology has two major differences when compared with popular methods used to estimate the solutions of PDEs. First, this methodology does not need to discretize the domain into a grid, rather, this methodology can randomly sample points from the domain during the training phase. Second, after training, this methodology produces an accurate analytical approximation of the solution throughout the entire training domain. Because the methodology produces an analytical solution, it is straightforward to obtain the solution at any point within the domain and to perform further manipulation if needed, such as differentiation. In contrast, other popular methods require extra numerical techniques if the estimated solution is desired at points that do not lie on the discretized grid, or if further manipulation to the estimated solution must be performed.
This study introduces a procedure to obtain all interpolating functions, y = f (x), subject to li... more This study introduces a procedure to obtain all interpolating functions, y = f (x), subject to linear constraints on the function and its derivatives defined at specified values. The paper first shows how to express these interpolating functions passing through a single point in three distinct ways: linear, additive, and rational. Then, using the additive formalism, interpolating functions with linear constraints on one, two, and n points are introduced as well as those satisfying relative constraints. In particular, for expressions passing through n points, a generalization of the Waring's interpolation form is introduced. An alternative approach to derive additive constraint interpolating expressions is introduced requiring the inversion of a matrix with dimensions equally the number of constraints. Finally, continuous and discontinuous interpolating periodic functions passing through a set of points with specified periods are provided. This theory has already been applied to obtain least-squares solutions of initial and boundary value problems applied to nonhomogeneous linear differential equations with nonconstant coefficients.
We present a novel, accurate, fast, and robust physics-informed neural network method for solving... more We present a novel, accurate, fast, and robust physics-informed neural network method for solving problems involving differential equations (DEs), called Extreme Theory of Functional Connections, or X-TFC. The proposed method is a synergy of two recently developed frameworks for solving problems involving DEs: the Theory of Functional Connections TFC, and the Physics-Informed Neural Networks PINN. Here, the latent solution of the DEs is approximated by a TFC constrained expression that employs a Neural Network (NN) as the free-function. The TFC approximated solution form always analytically satisfies the constraints of the DE, while maintaining a NN with unconstrained parameters. X-TFC uses a single-layer NN trained via the Extreme Learning Machine (ELM) algorithm. This choice is based on the approximating properties of the ELM algorithm that reduces the training of the network to a simple least-squares, because the only trainable parameters are the output weights. The proposed methodology was tested over a wide range of problems including the approximation of solutions to linear and nonlinear ordinary DEs (ODEs), systems of ODEs, and partial DEs (PDEs). The results show that, for most of the problems considered, X-TFC achieves high accuracy with low computational time, even for large scale PDEs, without suffering the curse of dimensionality.
This paper shows that the Theory of Functional Connections (TFC) can be used to find the thrust (... more This paper shows that the Theory of Functional Connections (TFC) can be used to find the thrust (given by a sub-optimal linear control law) required to create and maintain periodic orbits, like resonant orbits with the rotation of the Earth. Thus, the satellite will be located at the same position in the sky over the surface of the Earth every day, despite the perturbations. This facilitates the predictions of its motion and, thus, its tracking. The perturbations considered are: the 𝐽 2 term of the Earth's gravitational potential and the gravitational presence of the Sun and the Moon. The proposed approach is applied to two distinct periodic orbits. The first is an 8 h periodic orbit while the second is a 24 h geosynchronous periodic orbit. The fuel costs to maintain periodicity are evaluated and compared with other two methods: in the first method the perturbations are perfectly canceled by a variable thrust and in the second method a two-impulsive maneuver is applied as also obtained using TFC. Comparisons with the other two methods show that the one proposed in this paper can be used to lower the fuel costs.
This study provides a least-squares-based numerical approach to estimate the boundary value geode... more This study provides a least-squares-based numerical approach to estimate the boundary value geodesic trajectory and associated parametric velocity on curved surfaces. The approach is based on the Theory of Functional Connections, an analytical framework to perform functional interpolation. Numerical examples are provided for a set of two-dimensional quadrics, including ellipsoid, elliptic hyperboloid, elliptic paraboloid, hyperbolic paraboloid, torus, one-sheeted hyperboloid, Moëbius strips, as well as on a generic surface. The estimated geodesic solutions for the tested surfaces are obtained with residuals at the machine-error level. In principle, the proposed approach can be applied to solve boundary value problems in more complex scenarios, such as on Riemannian manifolds.
This work presents a methodology to derive analytical functionals, with embedded linear constrain... more This work presents a methodology to derive analytical functionals, with embedded linear constraints among the components of a vector (e.g., coordinates) that is a function a single variable (e.g., time). This work prepares the background necessary for the indirect solution of optimal control problems via the application of the Pontryagin Maximum Principle. The methodology presented is part of the univariate Theory of Functional Connections that has been developed to solve constrained optimization problems. To increase the clarity and practical aspects of the proposed method, the work is mostly presented via examples of applications rather than via rigorous mathematical definitions and proofs.
In this paper, we propose a unified approach to solve the time-energy optimal landing problem on ... more In this paper, we propose a unified approach to solve the time-energy optimal landing problem on planetary bodies (e.g. planets, moons, and asteroids). In particular, the indirect optimization method, based on the derivation of the first order necessary conditions from the Hamiltonian, is exploited and the Two-Point Boundary Value Problem arising from the application of the Pontryagin Minimum Principle is solved using the Theory of Functional Connections. The optimal landing trajectories are accurately computed with a computational time on the order of 10-100 ms, using a MATLAB implementation. The speed and accuracy of the proposed method makes it suitable for real time applications. The algorithm is applied and validated for the landing on large (Mars and Moon) and small (asteroids Gaspra and Bennu) planetary bodies.
This study extends the functional interpolation framework, introduced by the Theory of Functional... more This study extends the functional interpolation framework, introduced by the Theory of Functional Connections, initially introduced for functions, derivatives, integrals, components, and any linear combination of them, to constraints made of shear-type and/or mixed derivatives. The main motivation comes from differential equations, often appearing in fluid dynamics and structures/materials problems that are subject to shear-type and/or mixed boundary derivatives constraints. This is performed by replacing these boundary constraints with equivalent constraints, obtained using indefinite integrals. In addition, this study also shows how to validate the constraints' consistency when the problem involves the unknown constants of integrations generated by indefinite integrations.
The theory of functional connections, an analytical framework generalizing interpolation, was ext... more The theory of functional connections, an analytical framework generalizing interpolation, was extended and applied in the context of fractional-order operators (integrals and derivatives). The extension was performed and presented for univariate functions, with the aim of determining the whole set of functions satisfying some constraints expressed in terms of integrals and derivatives of non-integer order. The objective of these expressions was to solve fractional differential equations or other problems subject to fractional constraints. Although this work focused on the Riemann-Liouville definitions, the method is, however, more general, and it can be applied with different definitions of fractional operators just by changing the way they are computed. Three examples are provided showing, step by step, how to apply this extension for: (1) one constraint in terms of a fractional derivative, (2) three constraints (a function, a fractional derivative, and an integral), and (3) two constraints expressed in terms of linear combinations of fractional derivatives and integrals.
This study shows how the Theory of Functional Connections (TFC) allows us to obtain fast and high... more This study shows how the Theory of Functional Connections (TFC) allows us to obtain fast and highly accurate solutions to linear ODEs involving integrals. Integrals can be constraints and/or terms of the differential equations (e.g., ordinary integro-differential equations). This study first summarizes TFC, a mathematical procedure to obtain constrained expressions. These are functionals representing all functions satisfying a set of linear constraints. These functionals contain a free function, g(x), representing the unknown function to optimize. Two numerical approaches are shown to numerically estimate g(x). The first models g(x) as a linear combination of a set of basis functions, such as Chebyshev or Legendre orthogonal polynomials, while the second models g(x) as a neural network. Meaningful problems are provided. In all numerical problems, the proposed method produces very fast and accurate solutions.
This paper introduces an efficient approach to solve quadratic and nonlinear programming problems... more This paper introduces an efficient approach to solve quadratic and nonlinear programming problems subject to linear equality constraints via the theory of functional connections. This is done without using the traditional Lagrange multiplier technique. In particular, two distinct expressions (fully satisfying the equality constraints) are provided, to first solve the constrained quadratic programming problem as an unconstrained one for closed-form solution. Such expressions are derived by utilizing an optimization variable vector, which is called the free vector g by the theory of functional connections. In the spirit of this theory, for the equality constrained nonlinear programming problem, its solution is obtained by the Newton's method combining with elimination scheme in optimization. Convergence analysis is supported by a numerical example for the proposed approach.
In this paper, we consider several new applications of the recently introduced mathematical frame... more In this paper, we consider several new applications of the recently introduced mathematical framework of the Theory of Connections (ToC). This framework transforms constrained problems into unconstrained problems by introducing constraint-free variables. Using this transformation, various ordinary differential equations (ODEs), partial differential equations (PDEs) and variational problems can be formulated where the constraints are always satisfied. The resulting equations can then be easily solved by introducing a global basis function set (e.g., Chebyshev, Legendre, etc.) and minimizing a residual at pre-defined collocation points. In this paper, we highlight the utility of ToC by introducing various problems that can be solved using this framework including: (1) analytical linear constraint optimization; (2) the brachistochrone problem; (3) over-constrained differential equations; (4) inequality constraints; and (5) triangular domains.
This work considers fractional operators (derivatives and integrals) as surfaces f (x, α) subject... more This work considers fractional operators (derivatives and integrals) as surfaces f (x, α) subject to the function constraints defined by integer operators, which is a mandatory requirement of any fractional operator definition. In this respect, the problem can be seen as the problem of generating a surface constrained at some positive integer values of α for fractional derivatives and at some negative integer values for fractional integrals. This paper shows that by using the Theory of Functional Connections, all (past, present, and future) fractional operators can be approximated at a high level of accuracy by smooth surfaces and with no continuity issues. This practical approach provides a simple and unified tool to simulate nonlocal fractional operators that are usually defined by infinite series and/or complicated integrals.
This article introduces physics-informed neural networks (PINNs) to the field of motion planning ... more This article introduces physics-informed neural networks (PINNs) to the field of motion planning by utilizing a PINN framework as the steering function in the kinodynamic rapidly-exploring random tree (RRT*) algorithm. The goal of this paper is to show that PINN-based methods can be used successfully for aerospace motion planning applications. We test the RRT* algorithm coupled with PINN steering, what we call PINN-RRT*, by solving spacecraft energy-optimal motion planning problems governed by the Hill-Clohessy-Wiltshire (HCW) equations of motion and nonlinear equations of relative motion (NERM), where a deputy satellite must rendezvous with a chief satellite while avoiding spherical keep-out-zones and complying with an approach corridor. The particular PINN framework we employ approximates the solution of nonlinear two-point boundary value problems (TPBVPs), which must be solved to form connections between waypoints in the RRT* tree, via the Theory of Functional Connections (TFC). TFC enables the PINN to analytically satisfy the boundary conditions (BCs) of the TPBVP. Thus, the admissible solution search space of each nonlinear TPBVP is reduced to just the trajectories that already satisfy the BCs. Using our proposed approach, each energy-optimal TPBVP solution during the run-time of the PINN-RRT* algorithm was computed in centiseconds and with an average error on the order of machine epsilon for both the HCW and NERM dynamics.
Uploads
Papers by Daniele Mortari
motion planning by utilizing a PINN framework as the steering function in the kino-
dynamic rapidly-exploring random tree (RRT*) algorithm. The goal of this paper
is to show that PINN-based methods can be used successfully for aerospace motion
planning applications. We test the RRT* algorithm coupled with PINN steering,
what we call PINN-RRT*, by solving spacecraft energy-optimal motion planning
problems governed by the Hill–Clohessy–Wiltshire (HCW) equations of motion and
nonlinear equations of relative motion (NERM), where a deputy satellite must ren-
rendezvous with a chief satellite while avoiding spherical keep-out-zones and complying with an approach corridor. The particular PINN framework we employ approximates the solution of nonlinear two-point boundary value problems (TPBVPs),
which must be solved to form connections between waypoints in the RRT* tree, via
the Theory of Functional Connections (TFC). TFC enables the PINN to analytically
satisfy the boundary conditions (BCs) of the TPBVP. Thus, the admissible solution
search space of each nonlinear TPBVP is reduced to just the trajectories that already
satisfy the BCs. Using our proposed approach, each energy-optimal TPBVP solution
during the run-time of the PINN-RRT* algorithm was computed in centiseconds
and with an average error on the order of machine epsilon for both the HCW and
NERM dynamics.
motion planning by utilizing a PINN framework as the steering function in the kino-
dynamic rapidly-exploring random tree (RRT*) algorithm. The goal of this paper
is to show that PINN-based methods can be used successfully for aerospace motion
planning applications. We test the RRT* algorithm coupled with PINN steering,
what we call PINN-RRT*, by solving spacecraft energy-optimal motion planning
problems governed by the Hill–Clohessy–Wiltshire (HCW) equations of motion and
nonlinear equations of relative motion (NERM), where a deputy satellite must ren-
rendezvous with a chief satellite while avoiding spherical keep-out-zones and complying with an approach corridor. The particular PINN framework we employ approximates the solution of nonlinear two-point boundary value problems (TPBVPs),
which must be solved to form connections between waypoints in the RRT* tree, via
the Theory of Functional Connections (TFC). TFC enables the PINN to analytically
satisfy the boundary conditions (BCs) of the TPBVP. Thus, the admissible solution
search space of each nonlinear TPBVP is reduced to just the trajectories that already
satisfy the BCs. Using our proposed approach, each energy-optimal TPBVP solution
during the run-time of the PINN-RRT* algorithm was computed in centiseconds
and with an average error on the order of machine epsilon for both the HCW and
NERM dynamics.