FOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE Symmetric Matrix Properties and Applications: A Guide A symmetric matrix is one that is equal to its transpose. Here's how to identify one and how it can be applied. @ Written by Xichu Zhang rr EXPERT ri Image: Shutterstock / Built In Post @ Share=) JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE properties from the symmetry, we can find the most intuitive explanation of a symmetric matrix in its eigenvectors, which will give us a deeper understanding of the symmetric matrices. WHAT IS A SYMMETRIC MATRIX? ‘Asymmetric matrix is a matrix that is equal to its transpose. They contain three properties, including: Real eigenvalues, eigenvectors corresponding to the eigenvalues that are orthogonal and the matrix must be diagonalizable. A trivial example is the identity matrix. A non-trivial example can be something like: Ve mt ones ee ‘An example of a symmetric matrix. Image: Xichu Zhang While the definition may be as simple as that, they contain a lot of probabilities and can mean different things. I'll walk you through the important properties, explain them and then introduce the applications for a symmetric matrix. The Hermitian matrix is a complex extension of the symmetric matrix, which means in a Hermitian matrix, all the entries satisfy the following: aj = ij Definition 01. | Image: Xichu Zhang Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE Lo i il Example of the Hermitian matrix.| Image: xichu Zhang In data science, we mostly encounter matrices with real entries, since we are dealing with real-world problems. So, we'll be exploring real cases of symmetric matrices. Properties of a Symmetric Matrix Three properties of symmetric matrices are introduced in this section. They are considered to be the most important because they concern the behavior of eigenvalues and eigenvectors of those matrices. Those are the fundamental characteristics, which distinguishes symmetric matrices from non-symmetric ones. SYMMETRIC MATRIX PROPERTIES 1. Has real eigenvalues 2. Kigenvectors corresponding to the eigenvalues are orthogonal 3. Always diagonalizable PROPERTY 1: SYMMETRIC MATRICES HAVE REAL EIGENVALUES. pot (@) shareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE The proportions that the vectors are scaled are called eigenvalues. We denote them by . Therefore, we have the relation ax = Ax. The proof is fairly easy, but it requires some knowledge of linear algebra. So, we will still go through it step by step. We'll start with ax = ax by the conjugate transpose of x, x", and we arrive at this equation: ct Ag =z" \r Equation 12, |Image: Xichu Zhang It’s important to note that A is a scalar, which means the multiplication involving Ais commutative. Therefore, we can move it to the left of x#. zl Ar = dr" ac Equation 13. | Image: Xichu Zhang x"x is a Euclidean norm (or two-norm), which is defined as follows: U2llas= xt + -+28 Equation 14: A Euclidean norm. | Image: Xichu Zhang Ina two-dimensional Euclidean space, it’s the length of a vector with coordinates (x1, sy Xx). We can then write equationt.3 as: z4 Ar = All|} Equation 15. | Image: Xichu Zhang Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE (ax)" will leave us the complex conjugate of A, which we will denote it as A with a bar. der = \lx|3 Equation 17, |Image: Xichu Zhang We have seen x"x before in equation 1.3, After substituting in the two-norm, which we can do because both A_bar and x*x are scalars, we get this new equation: Mel = Alle Equation 18. Image: Xichu Zhang This leads to A, and its complex conjugate being equal. iN. Equation 19 | Image: Xichu Zhang Only under one circumstance is equation 1.9 valid, which is when Ais real. And that finishes the proof. MORE ON DaTA SCIENCE Recursive SQL Expression Visually Explained PROPERTY 2: EIGENVECTORS CORRESPONDING TO THE EIGENVALUES ARE ORTHOGONAL. Thi: is also a direct formal proof, but it’s simple. First, we need to recognize our target, pot (@) share which is the following:FOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE (Ax)? =a7 AP =a = aT A (Aaa)? = a3 AT =doxt = aT A Equation 1.1 top); Equation 1:2 (bottom), | Image: Xichu Zhang Now we need to approach equation 1.10. Let’s try to put x1 and x2 together. To do so, we multiply (ax.)* by x17 on the left side: r{ (Ag) = 2] Avira = Age} ta = = (since the result is a scalar) (x? Ay)? = =A ‘v= af Ary Equation 1.13. | Image: Xichu Zhang In equation 1.13 apart from the property of symmetric matrix, two other facts are used: 1. The matrix multiplication is associative (vectors are n by 1 matrix) 2. Matrix-scalar multiplication is commutative meaning we can move the scalar freely. Since dot production is commutative, this means x,'x, and x'x, are the same things. We now have: Aixa + Xo = Nox + Xo Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE denoted as N(A-Az) , when one eigenvector corresponds to multiple eigenvectors, N(A-AT) has a dimension larger than one. In this case, we have infinite choices for those eigenvectors, and we can always choose them to be orthogonal. There are cases where a real matrix has complex eigenvalues. This happens in rotation matrices. Why is that? Let Q be a rotation matrix. We know that the eigenvectors don’t change direction after being applied to Q. But if Q is a rotation matrix, how can x not change direction if x is a non-zero vector? The conclusion is that the eigenvector must be complex. Arotation matrix R(@) in the two-dimensional space is shown as follows: cos@ — sind RG) = [eee = Rotation matrix.| Image: Xichu Zhang R(@) rotates a vector counterclockwise by an angle . It is a real matrix with complex eigenvalues and eigenvectors. PROPERTY 3: SYMMETRIC MATRICES ARE ALWAYS DIAGONALIZABLE. This is known as the spectral theorem. It is also related to the other two properties of symmetric matrices, The name of this theorem might be confusing. In fact, the set of all the eigenvalues of a matrix is called a spectrum. Also, we can think about it like this: Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE Figure 115 comparing an un-stretched vector (left} to a stretched vector (right). | Image: Xichu Zhang ‘A matrix that is diagonalizable means there exists a diagonal matrix D (all the entries outside of the diagonal are zeros), such that p-sap = 0, where P is an invertible matrix. We can also say that a matrix is diagonalizable if the matrix can be written in the form A = pop-s, ‘The decomposition is unique up to the permutation of the entries on the diagonal in D and the scalar multiplication of the eigenvectors in P. We also need to note that the diagonalization is equivalent to finding eigenvectors and eigenvalues, no matter if the matrix is symmetric or not. However, for a non-symmetric matrix, D doesn’t have to be an orthogonal matrix, ‘Those two definitions are equivalent, but they can be interpreted differently. This decomposition makes raising the matrix to power very handy. The second one, A = pop->, tells us how A can be decomposed. Meanwhile, the first one, P-*aP = D, is the one that tells us A can be diagonalized. It tells us that it is possible to align the standard basis given by the identity matrix with the eigenvectors. The orthogonality of the eigenvectors, which is shown in property two, enables this. Post) shareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE vector back to the standard basis. From the perspective of the vector, the coordinate system is aligned with the standard basis with the eigenvectors. z 5% Hee \ P eed Figure 116 alignment of the basis. | Image: Xichu Zhang This alignment is shown in the figure above. The matrix used in this example is: 2 2 =F 1 2 O|=V -10 °5 5.2534 Equation 117 for figure 116. | mage: Xichu Zhang Where V is the matrix with eigenvectors of length one as the columns, each of them corresponds to the eigenvalues in the diagonal matrix. As for the calculation, we can let eig in MATLAB do the work. This provertv follows the svectral theorem directly. which dictates that if A is Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE eigenvectors). We can find another eigenvector in n-dimensional space, which is orthogonal to those n-1 dimensional eigenvectors. Thus, the n-by-n matrix is also diagonalizable. See all Data + Analytics jobs at }op tech companies & startups VIEW 30BS How to Identify the Definiteness of a Symmetric Matrix When are those properties useful? Even before the formal study of matrices, they have been used for solving systems of linear equations for a long time. Think about the matrices as operators, The information of the linear equations is stored in those operators, meaning matrices can be used to study the behavior of functions. Beyond symmetry, an even better property matrix can have is positive-definiteness. If a symmetric (or Hermitian) matrix is positive-definite, all of its eigenvalues are positive. Ifall of its eigenvalues are non-negative, then it is a semi-definite matrix, For a matrix to be positive-definite. it’s reauired to be symmetrical because of property post (@) share= JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE Video explaining the basics of a symmetric matrix | Video: Don't Memorise Symmetric vs. Skew-Symmetric Matrix Asymmetric matrix is a matrix equal to its transpose. In contrast, a skew-symmetric (or antisymmetric or antimetric) matrix is one that is opposite to its transpose, or when its transpose equals its negative. In a skew-symmetric matrix, the condition at = -a is met, plus all main diagonal entries are zero and the matrix’s trace equals zero. The sum of two skew-symmetric matrices, as well as a scalar multiple of a skew-symmetric matrix, will always be skew- symmetric.FOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE MACHINE LEARNING AND DATA SCIENCE ince matrices, are often s, such as Hessian matrices and cova Symmetric matric applied in machine learning and data science algorithms to optimize model processing and output. A matrix can help algorithms effectively store and analyze numerical data values, which are used to solve linear equations. ENGINEERING AND ROBOTICS Symmetrical matrices can provide structure for data points and help solve large sets of linear equations needed to calculate motion, dynamics or other forces in machines. This can be helpful for areas in control system design and optimization design of engineered systems, as well as for solving engineering problems related to control theory. How to Apply a Symmetric Matrix A good application of a symmetric matrix is the Hessian matrix. We will use this as an example to demonstrate using matrices for analyzing function behavior. When we try to find a local extreme, it’s always good to find out that the Hessian is positive definite. pot (@) shareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE And we call H(x) the Hessian of t, which 1s an n-by-n matrix. In detimition 1.18, the Hessian is written very compactly. It is the same thing as Aw a ony Fink, Rh Oky Hae werk fe ge Zs ea : a Frm Rak Br Definition 22 Hessian matrix. Image: Xichu Zhang How does this affect function behavior? We will look at one super simple example. Consider the function f(x, y) = y2-x. The Hessian is computed as follows: How) = Ee ae, aly | ay OF Equation 2.3 for a Hessian matrix. | Image: Xichu Zhang It can also be computed using the function hessian in MATLAB. Since it’s a diagonal matrix and the trace (sum of the entries on the diagonal) equals the sum of eigenvectors, we can immediately see that one of the eigenvalues is two and another one is negative two. They correspond to the eigenvectors v. = [1, @]” and va = [@, 1]*. This matrix is symmetric but not positive definite. Therefore, there's no Post @ ShareFOR EMPLOYERS JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN (©) REMOTE Figure 2.4 image of a function generated with MATLAB. | Image: Xichu Zhang ‘The image is generated using MATLAB with the following code. cle; clear; close all Xx = 5: 0.2: 55 < “5: 0.2: 55 [x y] = meshgrid(xx, yy); zz = x.92 ~ y.A25 figure mesh(x, y, 22) Post @ Share=) JOBS «COMPANIES = ARTICLES Vs SALARIESY—LEARN graph looks like this: Fi jure 25.| Image: Xichu Zhang | MORE ON DATA SCIENCE Understanding and Building Neural Network (NN) Models Symmetric Matrix Advantages FOR EMPLOYERS (©) REMOTE Matrices have wide-usage in numerous fields. When dealing with matrices, the concepts of positive-definiteness, eigenvectors, eigenvalues and symmetric matrices are often encemntered pot @) share