그림입니다.
원본 그림의 이름: Bigbook-Linearalgebra-cover.jpg
원본 그림의 크기: 가로 587pixel, 세로 798pixel           그림입니다.
원본 그림의 이름: sglee-4.GIF
원본 그림의 크기: 가로 292pixel, 세로 423pixel

 

 

 

 

Chaper 8






Diagonalization

 

 

 8.1 Matrix Representation of Linear Transformation

 8.2 Similarity and Diagonalization

 8.3 Diagonalization with orthogonal matrix,

 8.4 Quadratic forms

*8.5 Applications of Quadratic forms

 8.6 SVD and Pseudo-Inverse

 8.7 Complex eigenvalues and eigenvectors

 8.8 Hermitian, Unitary, Normal Matrices

*8.9 Linear system of differential equations

Exercises



In Chapter 6, we have studied how to represent a linear transformation from into as a matrix using its corresponding standard matrix.

We were able to compute the standard matrix of the linear transformation based on the fact that

   every vector in or can be expressed as a linear combination of the standard basis vectors.


In this chapter, we study how to represent a linear transformation from to with respect to arbitrary ordered bases for and .

In addition, we study relationship between different matrix representations of a linear transformation from to itself using transition matrices.

We also study matrix diagonalization.


Further, we study spectral properties of symmetric matrices and show that every symmetric matrix is orthogonally diagonalizable.


* A quadratic form is a quadratic equation which we come across in mathematics, physics, economics, statistics, and image processing, etc.

Symmetric matrices play a significant role in the study of quadratic forms.

We will learn how orthogonal diagonalization of symmetric matrices is used in the study of quadratic forms.


We introduce one of the most important concept in matrix theory called the singular value decomposition (SVD) which has many applications in science and engineering.


We will generalize matrix diagonalization of matrices and study least squares solutions and a pseudoinverse.


Furthur we deal with complex matrices having complex eigenvalues and eigenvectors.

We also introduce Hermitian matrices and unitary matrices that are complex counterparts corresponding to symmetric matrices and orthogonal matrices respectively.

Lastly, we study diagonalization of complex matrices.



8.1 Matrix Representation

 

 Lecture Movie : https://youtu.be/V_TBd7O_a70     http://youtu.be/jfMcPoso6g4

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-1.html

그림입니다.
원본 그림의 이름: mem000007b47fa7.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

In Chapter 6, we have studied how to represent a linear transformation from into as a matrix using the standard bases for and

    In this section, we find a matrix representation of a linear transformation from into with respect to arbitrary ordered bases for and .




Matrix Representation Relative to the Standard Bases

 

 

그림입니다.


Matrix Representation Using Arbitrary Ordered Bases for and

그림입니다.


Theorem

 8.1.1

Let be a linear transformation, and let

 

                      

 

be ordered bases for and respectively. Let . Then

 

                         ,

 

where the matrix representation of with respect to the ordered bases and is

                   .

Note that the matrix is called the matrix associated with the linear transformation with respect to the bases and .



 Recall that any vector can be uniquely represented as a linear combination of vectors in , say


                             .


       Then the coordinate vector for relative to the basis is

                                 .


       By the linearity of , we have

       . Since is a vector in , the coordinate vector of relative to satisfies


          

               .                              



Thus the matrix is the matrix whose -th column is the coordinate vector of with respect to the basis .


[Remarks]

 

 

 

 

 

(1) By Theorem 8.1.1 we can compute  by a matrix-vector multiplication, that is,

 

                      .

 

(2) The matrix varies according the ordered bases .

       For example, if we change the order of the vectors in the ordered base , then the columns of change as well.

(3) The matrices and are distinct, but they have the following relationship:

                .

 

(4) If and , then is denoted by and

        is called the matrix representation of relative to the ordered basis .

 

 

 

 

 


Define a linear transformation via and let

        ,

be ordered bases for and respectively. Compute .


Since , we first compute  .:


            , ,

We now find the coordinate vectors of the above vectors relative to the ordered basis . Since


   

   

   ,


we need to solve the corresponding linear systems with the same coefficient matrix .

  The augmented matrix for all of the three linear systems is . By converting this into its RREF, we get


                      .


Therefore, , , and hence

   .

   (Note that .)                                           


Let be defined via and

   

be ordered bases for and respectively. Find .


Since , , we have


   .


We can get its RREF as follows:  


  
                        .


Therefore, we get as


                   .                  



① Write .

                                                                     

x, y = var('x, y')

h(x, y) = [x+y, x-3*y, -2*x+y]

T = linear_transformation(QQ^2, QQ^3, h)

x1=vector([1, 1])

x2=vector([2, 1])

y1=vector([1, 0, -1])

y2=vector([-1, 2, 1])

y3=vector([0, 1, 1])

B=column_matrix([y1, y2, y3, T(x1), T(x2)])  # Matrix whose columns are  the vectors defined above

print B

                                                                    

[ 1 -1  0  2  3]

[ 0  2  1 -2 -1]

[-1  1  1 -1 -3]


② RREF

                                                                     

C=B.echelon_form()

print C

                                                                    

[   1    0    0  1/2  5/2]

[   0    1    0 -3/2 -1/2]

[   0    0    1    1    0]


③ Finding

                                                                     

A=C.submatrix(0, 3, 3, 2)  # C.submatrix(a, b, c, d)

                                       # submatrix with c consecutive rows of C starting from row a+1 and d 

                                      # consecutive columns of C starting from  column b+1

print A

                                                                    

[ 1/2  5/2]

[-3/2 -1/2]

[   1    0]    


We shall include calculation using the inbuilt function. Following are the codes.

                                                                     

var('x,y')

h(x,y)=[x+y,x-3*y,-2*x+y]

V=QQ^2;W=QQ^3

T=linear_transformation(V,W,h)

y1=vector(QQ,[1,1]);y2=vector(QQ,[2,1])

x1=vector(QQ, [1,0,-1]);x2=vector(QQ, [-1,2,1]);x3=vector(QQ,[0,1,1]);

alpha=[y1,y2]; beta=[x1,x2,x3]

V1=V.subspace_with_basis(alpha); W1=W.subspace_with_basis(beta)

T1=(T.restrict_domain(V1)).restrict_codomain(W1)

T1.matrix(side='right')

                                                                    

[ 1/2  5/2]

[-3/2 -1/2]

[   1    0]                                               



Let be a linear transformation defined as and

       consider the ordered bases , for and respectively.

 

  Answer the following questions:


(1) Find .

(2) Compute using  in (1).

(3) Using the definition of , find the standard matrix and  ,

      where and are the standard bases of and respectively.


(1) Since , and

     where and

       . Hence  .


(2) Since , we have

   

               

   (Note that

                      .)


(3) , .           


 

 

 

 

 

Composition of Linear Transformations

     그림입니다.

Let be a linear transformation from a vector space with an ordered basis into a vector space with an ordered basis ,

      and be a linear transformation from a vector sapce with an ordered basis into a vector space with an ordered basis .

   Suppose these linear transformations have their corresponding matrix representations and respectively.

     We can consider the composition . Then its matrix representation is

 

                      ,

 

That is, the product of the two matrix representations of and .

 

 

 

 

 

 










[Remark]

Transition Matrix

 

 as

 

 

As we have discussed earlier, , the matrix is called the transition matrix from ordered basis to ordered basis .

   We can consider the transition matrix as linear transformation .

                  그림입니다.

 

 

 

 


Let be linear transformations defined as

                   and

respectively. Consider the ordered bases , , for .

  Find the matrix representation of the composition with respect to the ordered bases and .


                                                                     

x, y = var('x, y')

ht(x, y) = [2*x+y, x-y];hs(x, y) = [-x+5*y, 2*x+3*y]

T = linear_transformation(QQ^2, QQ^2, ht)

S = linear_transformation(QQ^2, QQ^2, hs)

x1=vector([1, 0]);x2=vector([0, 1]);x3=vector([1, 1])

B=column_matrix([x2, x1, T(x1), T(x2)])

C=B.echelon_form()

MT=C.submatrix(0, 2, 2, 2)

print "Matrix of T="

print MT

D=column_matrix([x1, x3, S(x2), S(x1)])

E=D.echelon_form()

MS=E.submatrix(0, 2, 2, 2)

print "Matrix of S="

print MS

print "MS*MT="

print MS*MT

                                                                    

Matrix of T=

[ 1 -1]

[ 2  1]

Matrix of S=

[ 2 -3]

[ 3  2]

MS*MT=

[-4 -5]

[ 7 -1]

                                                                     

F=column_matrix([x1, x3, S(T(x1)), S(T(x2))])

G=F.echelon_form()

MST=G.submatrix(0, 2, 2, 2)

print "Matrix of S*T="

print MST

                                                                    

Matrix of S*T=

[-4 -5]

[ 7 -1]                                                     



Consider  the linear transformation defined by

           and

respectively. Find the composition and the matrix associated with composition.

   Hence show that the matrix of the composition is a product of the matrices associated with and .


                                                                     

x, y, z,w = var('x, y, z,w')

output1 = [x+z+w,x+y+2*z-w,2*x+y+3*z-2*w]

output2 = [x+z,x+3*y+2*z,2*x-y+3*z,y-z]

T_symbolic(x, y, z,w)=output1

T=linear_transformation(QQ^4, QQ^3, T_symbolic)

S_symbolic(x, y, z)=output2

S=linear_transformation(QQ^3, QQ^4, S_symbolic)

A= T.matrix(side='right')

B= S.matrix(side='right')

print A

print B

U=S*T;  

C=U.matrix(side='right')

print C

B*A==C

                                                                    

[ 3  1  4 -1]

[ 8  5 13 -6]

[ 7  2  9 -3]

[-1  0 -1  1]

True                                                           

 



  8.2 Similarity and Diagonalization

 

 Lecture Movie : https://youtu.be/NHEyutjknyo  http://youtu.be/xirjNZ40kRk

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-2.html

그림입니다.
원본 그림의 이름: mem000007b40001.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

In this section, we present various matrix representations of a linear transformation from to itself in terms of transition matrix.

    We also study when the transition matrix becomes a diagonal matrix.


 

[Remark]

Relationship between matrix representations and

 

 

 

 

              .

 

 

 

 

 

 

   


Theorem

 8.2.1

Let be a linear transformation and and be ordered bases for .

 

      If , then we have

 

                           ,

 

where  is the transition matrix from to .


 

              그림입니다.
원본 그림의 이름: CLP000016340004.bmp
원본 그림의 크기: 가로 781pixel, 세로 281pixel

                    .


http://www.math.tamu.edu/~yvorobet/MATH304-503/Lect2-12web.pdf 


Let be a linear transformation defined by .

      If is the standard basis for and is a basis for , find using the transition matrix .


Let be the standard matrix relative to the standard basis for linear transformation .

        Then we can find . If  , then


                    .


Therefore, and by Theorem 8.2.1 we get as follows:


         .    

        

                                                                     

x, y = var('x, y')

h(x, y) = [2*x-y, x+3*y]

T = linear_transformation(QQ^2, QQ^2, h)

x1=vector([1, 0]);x2=vector([0, 1])

y1=vector([0, 1]);y2=vector([-1, 1])

B=column_matrix([x1, x2, y1, y2])

C=B.echelon_form()

P=C.submatrix(0, 2, 2, 2)

print "Transition Matrix="

print P

A = T.matrix(side='right') 

print "A="

print A

print "P.inverse()*A*P"

print P.inverse()*A*P

D=column_matrix([y1, y2, T(y1), T(y2)])

E=D.echelon_form()

print "Matrix of A wrt beta="

print E.submatrix(0, 2, 2, 2)

                                                                    

Transition Matrix=

[ 0 -1]

[ 1  1]

A=

[ 2 -1]

[ 1  3]

P.inverse()*A*P

[ 2 -1]

[ 1  3]

Matrix of A wrt beta=

[ 2 -1]

[ 1  3]                                                  





Similarity

   


Definition

 [Similarity]

 

 

 

 

For square matrices and of the same order, if there exists an invertible matrix such that

                              ,

then we say that is similar to . We denote it as .

 

 

 

 

  


For , , , it can be shown that  . Hence is similar to .                    


   

Theorem

 8.2.2

For square matrices of the same order, the following hold:

 (1)

 (2)

 (3)

Therefore, the similarity relation is an equivalence relation.

 

Theorem

 8.2.3

For square matrices of the same order, if are similar to each other, then we have the following:

 (1) .

 (2) .

 Since , there exists an invertible matrix such that .


       (1) By the multiplicative property of determinant,

       

           ()

           

           


       (2)    ( )

                                                          


 Since similar matrices have the same determinants, it follows that they have the same characteristic equations and hence the same eigenvalues.

    In ordered to solve problems of determinant and/or eigenvalues of a square matrix, we can use its similar matrices which make the problems simpler.



Diagonalizable Matrices

   

Definition

 [Diagonalizable Matrices]

 

 

 

 

Suppose a square matrix is similar to a diagonal matrix, that is, there exists an invertible matrix such that is a diagonal matrix. Then is called diagonalizable and the invertible matrix is called a diagonalizing matrix for .

 

 

 

 

  

 If , then and hence we have

   ( multiplications of )

      

     

 This implies that if a matrix is diagonalizable, then its powers can be very easily computed.



For invertible matrix and matrix , we have

.

Hence is diagonalizable.                                


http://matrix.skku.ac.kr/RPG_English/8-TF-diagonalizable.html 

그림입니다.
원본 그림의 이름: mem000007b40009.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

 

                                                                     

A=matrix(QQ, [[1, 1], [-2, 4]])

print A.is_diagonalizable()          # Checking if diagonalizable

                                                                    

True                                                     



Since every diagonal matrix satisfies , it is

diagonalizable.                                                  



Note that not every matrix is diagonalizable.



Show that is not diagonalizable. 


Suppose to the contrary that is diagonalizable, that is, there exist an invertible matrix and a diagonal matrix with


                      , , ,


such that . Since , we have


                           ,


which gives . Hence .


If , then and (). Hence . Similarly, we can show that .

   The conditions and give a contradiction to  . Therefore, is not diagonalizable.               



Equivalent Conditions for Diagonalizability


  

Theorem

 8.2.4 [Equivalent Condition]

Let be a square matrix of order . Then is diagonalizable if and only if has linearly independent eigenvectors. 

  Furthermore, if is diagonalizable, then is similar to diagonal matrix whose main diagonal entries are equal to

    the eigenvalues of , and the th column of a diagonalizing matrix is an eigenvector of corresponding to eigenvalue .


  If is diagonalizable, then there exists an invertible matrix


                           


       such that where . Since , we get .

   Hence are eigenvalues of and . Note that are eigenvectors of corresponding to eigenvalues  respectively.

     Since is invertible, it follows that its columns are linearly independent.


        Suppose has eigenvalues and their corresponding eigenvectors that are linearly independent. We define as


                      .

Then


    

               .

       Since the columns of are linearly independent, the matrix is invertible, giving . Therefore is diagonalizable.

                     


[Remark] 

 Procedure for diagonalizing a matrix

 

 

 

 

● Step 1: Find linearly independent eigenvectors of .

● Step 2: Construct a matrix whose columns are in this order.

● Step 3: The matrix diagonalize and is a diagonal matrix whose main diagonal entries are eigenvalues of

                         .

 

 

 

 



It can be shown that the matrix has eigenvalues and their corresponding eigenvectors are respectively.

     Since these eigenvectors are linearly independent, by Theorem 8.2.4, is diagonalizable.

     If , then we have


               .    



Show that is diagonalizable and find the diagonalizing matrix of .


 

                                                                     

A=matrix([[0, 0, -2], [1, 2, 1], [1, 0, 3]])

print A.eigenvalues()           # Eigenvalue Computation

                                                                    

[1, 2, 2]


 has eigenvalues . We now compute linearly independent eigenvectors of .


For , we solve (that is, ) for .

                                                                           

E=identity_matrix(3)

print (E-A).echelon_form()

                                                                    

[ 1  0  2]

[ 0  1 -1]

[ 0  0  0]


Since , we get ; For , we solve (that is, ) for .

                                                                           

print (2*E-A).echelon_form()

                                                                    

[1 0 1]

[0 0 0]

[0 0 0]


This gives  and hence

                      .

                                                                     

x1=vector([-2, 1, 1])

x2=vector([-1, 0, 1])

x3=vector([0, 1, 0])

P=column_matrix([x1, x2, x3])

print P

print

print P.det()

                                                                    

[-2 -1  0]

[ 1  0  1]

[ 1  1  0]


1

Since the above computation shows that the determinant of is not zero, is invertible.

    Hence its columns are linearly independent. Therefore, by Theorem 8.2.4, is diagonalizable.     

                                                                     

print P^-1*A*P # Computing diagonal matrix whose main diagonal entries  are eigenvalues of A.

                                                                    

[1 0 0]

[0 2 0]

[0 0 2]                                                           ■


   

Theorem

 8.2.5

If are eigenvectors of corresponding to distinct eigenvalues , then the set is linearly independent.

(Exercise) Hint. This can be proved by the mathematical induction on .


   

Theorem

 8.2.6

If a square matrix of order has distinct eigenvalues, then is diagonalizable.


 Let be eigenvectors of corresponding to distinct eigenvalues.

      Then, by Theorem 8.2.5, the eigenvectors are linearly independent. Therefore, Theorem 8.2.4 implies that is diagonalizable.                                                      


The matrix in Example 6 has two distinct eigenvalues. Thus, by Theorem 8.2.6, is diagonalizable.                    



Note that a diagonal matrix can have a repeated eigenvalue. Therefore, the converse of Theorem 8.2.6 is not necessarily true.



Algebraic Multiplicity and Geometric Multiplicity of an Eigenvalue


   

Definition

 [Algebraic and Geometric Multiplicity]

 

 

 

 

Let be distinct eigenvalues of . Then the characteristic polynomial of can be written as

그림입니다.

In the above expression the sum of the exponents is equal to . The positive integer is called the algebraic multiplicity of and the number of linearly independent eigenvectors corresponding to the eigenvalue is called the geometric multiplicity of .

 

 

 

 

  

   

Theorem

 8.2.7 [Equivalent Condition for Diagonalizability]

Let be a square matrix of order . Then is diagonalizable if and only if the sum of the geometric multiplicities of eigenvalues of is equal to .


By Theorem 8.2.4, an equivalent condition for a square matrix of order to diagonalizable is to have linearly independent eigenvectors.

      Since the sum of the geometric multiplicities of eigenvalues of is equal to the number of linearly independent eigenvectors of and it is equal to , the result follows.

                                                        


   

Theorem 

 8.2.8

Let be a square matrix and be an eigenvalue of . Then the algebraic multiplicity of is greater than or equal to the geometric multiplicity of .


 Let be the geometric multiplicity of an eigenvalue of , and let be the matrix whose columns are the linearly independent eigenvectors of corresponding to .

     We can construct an invertible matrix by adding linearly independent columns to . Let be the resulting invertible matrix and let be the inverse of .

     Then . Note that and have same characteristic polynomials. Since the first column of has have in its diagonal,

      the characteristic polynomial of has a factor of at least .

        Hence, the algebraic multiplicity of is greater than or equal to the geometric multiplicity of .                                         


Theorem 

 8.2.9 [Equivalent Condition for Diagonalizability]

Let be a square matrix of order .

    Then is diagonalizable if and only if each eigenvalue of has the same algebraic and geometric multiplicity.


If is diagonalizable, then there exists an invertible matrix and a diagonal matrix such that , or equivalently .

     This implies that times column of is equal to scalar multiple of the column of .

        Hence, all the columns of are eigenvectors of , which implies that each eigenvalue of has the same algebraic and geometric multiplicity.

 

     The converse follows from Theorem 8.2.7.       



For , the characteristic equation is   .

    Hence the eigenvalues of are and has algebraic multiplicity 2. 

     The following two vectors are linearly independent eigenvectors of


                         ,  


corresponding to and respectively.

However, matrix does not have three linearly independent eigenvectors.

  Hence Theorem 8.2.4 implies that is not diagonalizable.        


Note that the eigenvalue with algebraic multiplicity 2 has only one linearly independent eigenvector of .


It can be shown that has eigenvalues 3 and with algebraic multiplicity 1 and 2 respectively,

    We can further show that geometric multiplicity of 3 and 2 are 1 and 2 respectively.

    Hence is diagonalizable since has 3 linearly independent eigenvectors.

       It can be verified that diagonalize and , where . Let us further compute .

 

   




8.3 Diagonalization with orthogonal matrix,

 

*Function of matrix

 Lecture Movie : https://youtu.be/B8ESZZQIlzA http://youtu.be/jimlkBGAZfQ

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-3.html

그림입니다.
원본 그림의 이름: mem000007b40002.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

Symmetric matrices appear in many applications.

     In this section, we study useful properties of symmetric matrices and show that every symmetric matrix is orthogonally diagonalizable

    *Furthermore, we study matrix functions using matrix diagonalization.



Orthogonal Matrix

   

Definition

 [Orthogonal Matrix]

 

 

 

 

For a real square matrix , if is invertible and , then is called an orthogonal matrix.

 

 

 

 

     

Theorem

 8.3.1

If is an orthogonal matrix of order , then the following hold:

 

(1) The rows of are unit vectors and they are perpendicular to each other.

(2) The columns of are unit vectors and they are perpendicular to each other.

(3) is invertible.

(4) for any vector     (Norm Preserving Property).


Similar to the proof of Theorem 6.2.3.



For , we have . Since

   , is an orthogonal matrix.    



   

Definition

 [Orthogonal Similarity]

 

 

 

 

Let and be square matrices of the same order.

   If there exists an orthogonal matrix such that , then is said to be orthogonally similar to .

 

 

 

 

   

Definition

 [Orthogonally Diagonalizable]

 

 

 

 

For a square matrix , if there exists an orthogonal matrix diagonalizing ,

   then is called orthogonally diagonalizable and is called a matrix orthogonally diagonalizing .

 

 

 

 



 Next we try to answer the following question. What matrices are orthogonally diagonalizable? 


Theorem

 8.3.2

Every eigenvalue of a real symmetric matrix is a real number.

Let be an eigenvalue of and , an eigenvector corresponding to .

     Then . Now premultiplying the first equation both sides by to have .

      Taking the complex conjugate both sides, we have which implies .

         Now, we get  . Note that . Hence which means is real.                                                  


Note: Compare this proof with the one in Theorem 8.7.1


The symmetric matrix has characteristic equation  .

     Hence its eigenvalues are  which are all real numbers.            


   

Theorem

 8.3.3

If a square matrix is symmetric, then eigenvectors of corresponding to distinct eigenvalues are perpendicular to each other.

Let , be eigenvectors corresponding to distinct eigenvalue and () respectively.

      We have .

           Since , the above equation implies .                                        


   

Theorem

 8.3.4

Let be a square matrix. Then is orthogonally diagonalizable if and only if the matrix is symmetric.

 () Suppose is orthogonally diagonalizable. Then there exists an orthogonal matrix and a diagonal matrix such that .

         Since , we have

 

                      .

       Hence


       

                         

                          .

       Therefore, is symmetric. 


       () : Read : http://www.maths.manchester.ac.uk/~peter/MATH10212/notes10.pdf   


   

Theorem

 8.3.5

If is a symmetric matrix of order , then has eigenvectors forming an orthonormal set.

 Since is symmetric, by Theorem 8.3.4, is orthogonally diagonalizable,

       that is, there exists an orthogonal matrix and a diagonal matrix such that .

        Hence the main diagonal entries of are the eigenvalues of and the columns of are eigenvectors of .

        Since the columns of the orthogonal matrix form an orthonormal set, the eigenvectors of are orthonormal.     


   

Theorem

 8.3.6

For a square matrix of order , the following are equivalent:

 

(1) is orthogonally diagonalizable.

(2) has eigenvectors that are orthonormal.

(3) is symmetric.

 How to find an orthogonal matrix diagonalizing a given symmetric matrix ?



For symmetric matrix , find an orthogonal matrix diagonalizing .


 Since the characteristic equation of is , the eigenvalues of are ,.

      Note that all the eigenvalues are distinct. Hence there exist eigenvectors of that are orthogonal.

      The corresponding eigenvectors of are


                        .


By normalizing , we get an (real) orthogonal matrix diagonalizing :


        



It can be shown that the matrix has eigenvalues (algebraic multiplicity 2) and .

    Hence we need to check if has two linearly independent eigenvectors. After eigenvector computation, we get


                       


that are linearly independent eigenvectors corresponding to eigenvalue -3.

  Using the Gram-Schmidt Orthonormalization, we get


   , ,

   , .


We can find an eigenvector corresponding to the eigenvalue and normalization gives

                              .

Therefore, the orthogonal matrix  diagonalizing is given by  

 

                                .              



[Remark]

* Function of Matrices

 

 

 

 

● There are several techniques for extending a real function to a square matrix function such that interesting properties are maintained. You can find more the details on the following website:

https://en.wikipedia.org/wiki/Matrix_function 

http://youtu.be/B--ABwoKAN4

● http://www.siam.org/books/ot104/OT104HighamChapter1.pdf

 

 

 

 


                       그림입니다.
원본 그림의 이름: mem0000019c0001.JPG
원본 그림의 크기: 가로 217pixel, 세로 106pixel   

        그림입니다.
원본 그림의 이름: mem0000019c0002.png
원본 그림의 크기: 가로 560pixel, 세로 158pixel



8.4 Quadratic forms

 

 Lecture Movie : https://youtu.be/SxCLGhylkIk http://youtu.be/vWzHWEhAd-k

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-4.html

그림입니다.
원본 그림의 이름: mem000007b40003.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

A quadratic form is a polynomial each of whose terms is quadratic.

Quadratic forms appear in many scientific areas including mathematics, physics, economics, statistics, and image processing.

Symmetric matrices play an important role in analyzing quadratic forms.

In this section, we study how diagonalization of symmetric matrices can be applied to analyse quadratic forms.


   

Definition

 

 

 

 

 

An implicit equation in variables , for a quadratic curve can be expressed as

           .                      (1)

.

This can be rewritten in a matrix-vector form as follows:

           .                      (2)

 

 

 

 

  

[Remark] 

Graph for a quadratic curve (conic section)

 

 

 

 

The following are the types of conic sections:

Non-degenerate conic sections: Circle, Ellipse, Parabola, Hyperbola. See Figure 1 and Example 1.

Imaginary conic section: There are no points satisfying (1)

Degenerate conic section: The graph of the equation (1) is one point, one line, a pair of lines, or having no points. See Example 2.

 

 

 

 

 


                  그림입니다.

                                             Figure 1

    

[Remark] 

Conic Sections in the Standard Position

 

 

 

 

●     (Ellipse)                                     (3)

                그림입니다.
원본 그림의 이름: CLP000016f80002.bmp
원본 그림의 크기: 가로 784pixel, 세로 359pixel

 

●        or (Hyperbola)               (4)

                그림입니다.
원본 그림의 이름: CLP000016f80003.bmp
원본 그림의 크기: 가로 1080pixel, 세로 453pixel

 

●        or (Parabola, )                  (5)

 

              그림입니다.
원본 그림의 이름: CLP000016f80004.bmp
원본 그림의 크기: 가로 878pixel, 세로 289pixel

 

 

 

 

 


(Non-degenerate conic section)

Since the equation can be written as ,

    the graph of this equation is an ellipse.

      The equation has the standard form and

    hence its graph is a hyperbola. Since the equation can be put into , its graph is a parabola. 


                                                                     

var('x,y')

g1=implicit_plot(9*x^2+4*y^2-144==0,(x,-8,8),(y,-8,8),figsize=4,axes=True)

g2=implicit_plot(9*x^2-4*y^2+144==0,(x,-20,20),(y,-20,20),figsize=4,axes=True)

g3=implicit_plot(y^2+3*x==0,(x,-5,5),(y,-5,5),figsize=4,axes=True)

show(graphics_array([g1,g2,g3]))

                                                                    

           그림입니다.
원본 그림의 이름: sage0 (1).png
원본 그림의 크기: 가로 488pixel, 세로 316pixel 


(Degenerate conic section)

The graph of the equation is the -axis. The graph of consists of the two horizontal lines , .

  The graph of consists of the two lines and .

   The graph of consists of one point . The graph of has no points.      


 The graph of a quadratic equations with both and terms or both and terms is a translation of a conic section in the standard position.



Let us plot the graph of . By completing squares in , we get


                  .                       (6)


Hence by using the substitutions , we get

                              

in the -coordinate plane. This equation gives a hyperbola of the standard position in the -coordinate plane.

    Hence the graph of the equation (6) is obtained by translating the hyperbola in the standard position 3 units along the -axis and 1 unit along the -axis. 


 

                                                                     

var('x,y')

c1=implicit_plot(x^2/3-y^2/3-1==0,(x,-8,8),(y,-8,8),axes=True,figsize=5,color='red')

c2=implicit_plot((x-3)^2/3-(y-1)^2/3-1==0,(x,-8,8),(y,-8,8),axes=True,figsize=5,color='blue')

c1+c2

                                                                    

           그림입니다.
원본 그림의 이름: mem00000e740001.png
원본 그림의 크기: 가로 360pixel, 세로 359pixel            



Quadratic Form


   

Definition

 [Quadratic Form]

 

 

 

 

                                      (7)

 

is called the quadratic form of the quadratic equation (1).

 

 

 

 

   

The quadratic equations are quadratic forms, but the quadratic equation has a linear term and constant term 1 and hence it is not a quadratic form.  



 A quadratic form can be written in the form of . For example,

 or .


This means that the matrix above is not unique.


 We will use a symmetric matrix to write a quadratic forms:

            ,

            .


   We use symmetric matrices to represent quadratic forms because symmetric matrices are orthogonally diagonalizable. 


   

Definition

 

 

 

 

 

Let be a symmetric matrix of order and for real values .

  Then is called a quadratic form in .

 

 

 

 

  


 For a quadratic form in and , the -term is called a cross-product term.

 Using orthogonal diagonalization, we can eliminate the cross-product term.


 For a quadratic form

                           ,


  the matrix is symmetric, we can find orthonormal eigenvectors corresponding to the eigenvalues , of .

    The matrix is orthogonal and orthogonally diagonalizes , that is, .

   Since we can switch the roles of and by switching the roles of and , without loss of generality, we can assume .


  Therefore, we can consider as the rotation matrix in . Let for some . Then


                

                           


  and hence is a quadratic form without any cross-product term in the -coordinate system. Therefore, we get the following theorem.



Theorem

 8.4.1 [Diagonalization of a Quadratic Form]

Suppose a symmetric matrix has as its eigenvalues. Then, by rotating the coordinate axes, the quadratic form  can be written as follows in the -coordinate system

 

                     .                        (8)

 

If the determinant of is 1 and diagonalize , then the rotation can be obtained by or .



Using diagonalization of a quadratic form, determine which conic section the following quadratic equation describes.


               .                         (9)


The quadratic equation can be written as


.


Since the characteristic equation of the symmetric matrix is , the eigenvalues of are  .

   By Theorem 8.4.1, . Hence, in the new coordinate system, the quadratic equation becomes


       .

                           그림입니다.
원본 그림의 이름: CLP000016f80005.bmp
원본 그림의 크기: 가로 385pixel, 세로 379pixel

 

  Since eigenvectors corresponding to are

 

,

respectively, the orthogonal matrix diagonalizing is


 

           and   .


Therefore -coordinate axes are obtained by rotating the -axis 45° clockwise () and

  the equation (9) is an ellipse in the standard position relative to the -coordinate system.    


[Remark] 

Simulation for quadratic forms

 

 

 

 

http://www.geogebratube.org/student/m121534

 

   그림입니다.
원본 그림의 이름: CLP000008101ee9.bmp
원본 그림의 크기: 가로 900pixel, 세로 652pixel 그림입니다.
원본 그림의 이름: CLP000008100001.bmp
원본 그림의 크기: 가로 903pixel, 세로 653pixel

 

 

 

 

 



Sketch the graph of the following quadratic equation


         .              (10)


Letting , we can rewrite the equation (10) as follows:


         .                                (11)


Using rotation we first eliminate the cross-product terms. Since the characteristic equation of is

                  

                        


the eigenvalues of are , and their corresponding orthonormal eigenvectors are  , respectively.

    Hence we can take .


Using axis rotation , we get and and hence from (11), we obtain


              .                   (12)


We now use horizontal translation to remove -term in (12). By completing the squares in (12) we get


                        .


That is, (Note: ). Therefore, the equation (12) represents an ellipse in the -coordinate system                    


                                   

     

where the -coordinate system is obtained by  horizontally translating the -coordinate system 1 unit along the -axis.

                그림입니다.
원본 그림의 이름: CLP000016f80006.bmp
원본 그림의 크기: 가로 512pixel, 세로 376pixel



Surface in 3-dimensional space


 Let 


                       .                               (13)


Then, after diagonalization, we get 


                                                       (14)


in the rotated -coordinate system. This enables us to identify the graph of the equation (13) in .

 

In equation (14), if both , are positive, then the graph of equation (14) is a paraboloid opening upward (see figure (a) below). If both and are negative,

then the graph is a paraboloid opening downward (see figure (b) below).

Since the horizontal cross-section of each paraboloid is an ellipse, the above graphs are called elliptic paraboloids.


                          그림입니다.
원본 그림의 이름: CLP000016f80007.bmp
원본 그림의 크기: 가로 689pixel, 세로 366pixel

                                Elliptic Paraboloid


 In (14) if both of and are nonzero but have different signs, then  the graphs looks like a saddle in (a) and

      is called a  hyperbolic paraboloid. If one of and is zero, then the graph is parabolic cylinder in (b).


                       그림입니다.



Show that the graph of the following equation is an elliptic paraboloid and sketch its cross-section at .


                                        (15)


The quadratic form in (15) can be written as .

    We first find an orthogonal matrix diagonalizing the symmetric matrix .

      It can be shown that , and hence using , we can transform (15) into the following:


                   .                       (16)


The equation (16) represents an elliptic paraboloid in the -coordinate system.

   Note that the -coordinate system is obtained by rotating the -coordinate by angle counterclockwise. Hence, in , is given by


                    


and . Now we sketch the cross-section of equation (15) at .

     By substituting into (16), we get and hence the graph looks like the following:

                       그림입니다.
원본 그림의 이름: CLP000016f80009.bmp
원본 그림의 크기: 가로 408pixel, 세로 328pixel        


 Let's use Sage to graph equation (15)

                    

① Computing eigenvalues of

                                                                     

A=matrix(2, 2, [34, -12, -12, 41])

print A.eigenvalues()

                                                                    

[50, 25]


② Computing eigenvectors of

                                                                     

print A.eigenvectors_right()

                                                                    

[(50, [(1, -4/3)], 1),

(25, [(1, 3/4)], 1)]


③ Computing diagonalizing

                                                                     

G=matrix([[1, 3/4], [1, -4/3]])  # Constructing a matrix whose columns  are eigenvectors

P=matrix([1/G.row(j).norm()*G.row(j) for j in range(0,2)])

                                  # Normalizing the row vectors (The orthogonality follows from the

                                 # fact that the eigenvalues are distinct)

P=P.transpose()       # Constructing a matrix whose columns are

                               # orthonormal eigenvectors

print P

                                                                    

[ 4/5  3/5]

[ 3/5 –4/5]


④ Sketching two ellipses simultaneously

                                                                     

var('u, v')

s=vector([u, v])

B=P.transpose()*A*P

p1=implicit_plot(s*A*s==50, (u, -2, 2), (v, -2, 2), axes='true')

p2=implicit_plot(s*B*s==50, (u, -2, 2), (v, -2, 2), color='red', axes='true')

show(p1+p2)  # Ploting two graphs simultaneously

                                                                    

            그림입니다.
원본 그림의 이름: mem00000dd00001.png
원본 그림의 크기: 가로 585pixel, 세로 584pixel                     




8.5 *Applications of Quadratic forms

 

 Lecture Movie : http://youtu.be/cOW9qT64e0g

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-5.html

그림입니다.
원본 그림의 이름: mem000007b40004.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

By the theorem of principal axis (theorem 8.4.1), the graph of a 3D curve can be shown in the form of circle, ellipse or parabola in 2D.

 The specific shape is uniquely determined by signs of eigenvalues of the corresponding quadratic form.

    In this section, we define the sign of the quadratic form to identify the type of graph of given quadratic forms, and learn how to obtain the extrema of multivariable functions using them.



 Given a system of springs and masses, there will be one quadratic form that represents the kinetic energy of the system, and another which represents the potential energy of the system in position variables.

     Can one supply more details or omit this section. More details can be found in the following websites:


Application of Quadratic Forms and Sage:

        http://matrix.skku.ac.kr/2014-Album/Quadratic-form/

http://ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter32/section09.html 



               그림입니다.
원본 그림의 이름: mem0000019c0005.jpg
원본 그림의 크기: 가로 300pixel, 세로 300pixel



8.6 SVD and Pseudo-inverse

 

 Lecture Movie : https://youtu.be/0FYcU4DWhWQ https://youtu.be/ejCge6Zjf1M

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-6.html

그림입니다.
원본 그림의 이름: mem000007b40005.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

 We have learned that symmetric matrices are diagonalizable.

     We now extend the concept of diagonalization to matrices (not necessarily square or symmetric) resulting in a matrix decomposition and

     study pseudo-inverse and least square solutions using a matrix decomposition.


   

Theorem

 8.6.1 [Singular Value Decomposition]

Let be an real matrix. Then there exist orthogonal matrices of order and of order ,

    and an matrix such that 

 

                           ,                 (1)

 

where the main diagonal entries of are positive and listed in the monotonically decreasing order,

    and is a zero-matrix. That is,

 

    ,

       where .


   

Definition

 

 

 

 

 

Equation (1) is called the singular value decomposition (SVD) of .

  The main diagonal entries of the matrix are called the singular values of .

  In addition, the columns of are called the left singular vectors of and the columns of are called the right singular vectors of .

 

 

 

 


 The following theorem shows that matrices and are orthogonal matrices diagonalizing and respectively.

  

 

Theorem

  8.6.2

Let the decomposition be the singular value decomposition (SVD) of an matrix where are positive diagonal entries of . Then

 

 (1) .

 (2) .


Since , we get

       .

   Hence, by considering, and ,

 

  and


       respectively.                                                      



Find the SVD of .


 The eigenvalues of are and

     hence the singular values of are .

A unit eigenvector of corresponding to is , and a unit eigenvector of corresponding to

                is . We can also find unit eigenvectors of :

 

                .

Hence we get

            .

 

Therefore, the SVD of is

       .


① Computing the singular values of and eigenvectors of

                                                                     

A=matrix([[sqrt(3), 2], [0, sqrt(3)]])

B=A.transpose()*A

eig=B.eigenvalues()

sv=[sqrt(i) for i in eig]                # Computing singular values

print B.eigenvectors_right()           # Computing eigenvectors of

                                                                    

[(9, [(1, sqrt(3))], 1), (1, [(1, -1/3*sqrt(3))], 1)]


② Computing

                                                                     

G=matrix([[1, sqrt(3)], [1, -1/3*sqrt(3)]])

Vh=matrix([1/G.row(j).norm()*G.row(j) for j in range(0,2)])

Vh=Vh.simplify()    # Transpose of V

print Vh

                                                                    

[        1/2 1/2*sqrt(3)]

[1/2*sqrt(3)        -1/2]


③ Computing eigenvectors of

                                                                     

C=A*A.transpose()

print C.eigenvectors_right()          # Computing eigenvectors of

                                                                    

[(9, [(1, 1/3*sqrt(3))], 1), (1, [(1, -sqrt(3))], 1)]


④ Computing

                                                                     

F=matrix([[1, 1/3*sqrt(3)], [1, -sqrt(3)]])

U=matrix([1/F.row(j).norm()*F.row(j) for j in range(0,2)])

U=U.simplify().transpose()    # U

print U

                                                                    

[ 1/2*sqrt(3)          1/2]

[         1/2 –1/2*sqrt(3)]


⑤ Computing diagonal matrix

                                                                     

S=diagonal_matrix(sv); S

                                                                    

[3 0]

[0 1]


⑥ Verifying

                                                                     

U*S*Vh

                                                                    

[sqrt(3)       2]

[      0 sqrt(3)]                                                 

http://matrix.skku.ac.kr/2014-Album/MC.html    (SVD)          ■




Equivalent statement of invertible matrix on SVD


  

Theorem

 8.6.3

Let be an matrix. Then is a nonsingular matrix if and only if every singular value of is nonzero.


 Since , matrix is nonsingular if and only if is nonsingular.

     Hence, if is nonsingular, then all the eigenvalues of are nonzero.

      By Theorem 8.6.2, the singular values of are the square roots of the positive eigenvalues of .

       Hence the singular values of are nonzero.                      


   

Theorem

 8.6.4

Suppose are the singular values of an matrix . Then the matrix can be expressed as follows:

                           .                         (R)

The equation (R) is called a rank-one decomposition of .


Note that the pseudo-inverse of a matrix is important in the study of the least squares solutions for optimization problems.


 We can express an nonsingular matrix using the SVD


                               .                                 (2)


  Note that all of , , are nonsingular matrices and , are orthogonal matrices. Hence the inverse of can be expressed as


                               .                            (3)


If is not a square matrix or is singular, then (3) does not give an inverse of .

    However, we can construct a pseudo-inverse of by putting in (2) into the form (where is nonsingular).



 

Definition

 [Pseudo-Inverse]

 

 

 

 

For an matrix , the matrix is called a pseudo-inverse of ,

    where , are orthogonal matrices in the SVD of  and is

 

                    (where is nonsingular).

 

 

 

 

   

 We read as ‘dagger.’ If , then we define .



Truncated SVD

    http://langvillea.people.cofc.edu/DISSECTION-LAB/Emmie'sLSI-SVDModule/p5module.html



What is a truncated SVD?


We learned that singular value decomposition for any matrix so that .

     Let's take a closer look at the matrix . Remember is a diagonal matrix where

                                   and are the singular values of the matrix .

     A full rank decomposition of is usually denoted by where and are the matrices obtained by taking the first columns of and respectively.

    We can find a -rank approximation (or truncated SVD) to by taking only the first k largest singular values and the first k columns of U and V.



Find a pseudo-inverse of .


We first compute the (truncated) SVD1) of :

        http://matrix.skku.ac.kr/2014-Album/MC.html
   .

Then


  

            
      .
                                          


http://matrix.skku.ac.kr/RPG_English/8-MA-pseudo-inverse.html 

그림입니다.
원본 그림의 이름: mem000007b4000a.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

 

                                                                     

A=matrix(RDF,[[1,1],[0,1],[1,0]])

import numpy as np

Pinv=matrix(np.linalg.pinv(A))

Pinv

                                                                    

[ 0.333333333333 -0.333333333333  0.666666666667]

[ 0.333333333333  0.666666666667 -0.333333333333]



 If has , then is said to be of the full column rank.

   If has full column rank, then is nonsingular. If is nonsingular, then the pseudo-inverse of is equal to .


   

Theorem 

 8.6.5

If an matrix has full column rank, then the pseudo-inverse of is

 

                            .

 Let  be the SVD of . Then where is nonsingular. Then


                     .


      Since has full column rank, is nonsingular and matrix is an orthogonal matrix. Hence and


          

                     .     



Find the pseudo-inverse of using theorem 8.6.5.


       .


Since has full column rank,


    is nonsingular and

   .       


  

Theorem

 8.6.6

If is a pseudo-inverse of , then the following hold:

 

 (1)

 (2)

 (3)

 (4)

 (5)

 (6) .

(Exercise) https://en.wikipedia.org/wiki/Proofs_involving_the_Moore%E2%80%93Penrose_pseudoinverse




[Remark] 

 

 

 

 

 

A pseudo-inverse provides a tool for solving a least square problem. It is known that the least squares solution to the linear system is the solution to the normal equation . If has full column rank, then the matrix is nonsingular and hence

                     .

This means that if has full column rank, the least squares solution to is the pseudo-inverse times the vector .

 

 

 

 

 

 

 

  

Theorem 

 8.6.7

Let be an matrix and be a vector in . Then is the least squares solution to .


 Let be the SVD of with ( is nonsingular).

     Then and hence , Since


           

                      ,


       it follows that satisfies .                       



Find the least squares line passing through the four points  .


 

  Let be an equation for the line that fits to the points .

      Then, by letting , the given condition can be written as the linear system for which and .

  Since has full column rank, we get which is


   . Hence . Therefore, the least squares line is given by .                        

 

                                       그림입니다.
원본 그림의 이름: CLP00000d9c0001.bmp
원본 그림의 크기: 가로 633pixel, 세로 599pixel


var('x, y')

p1=plot(x + 3/2, x, -1,3, color='blue');

p2 = text("$x+ 3/2 $", (2,2), fontsize=20, color='blue')

show(p1+p2, ymax=4, ymin=-1)

in http://matrix.skku.ac.kr/Cal-Book/part1/CS-Sec-1-3.htm





8.7 Complex eigenvalues and eigenvectors

 

 Lecture Movie : https://youtu.be/ejGNmo9hhfI http://youtu.be/8_uNVj_OIAk

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-13-sec-8-7.html

그림입니다.
원본 그림의 이름: mem000007b40006.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

We have so far focused on real eigenvalues and real eigenvectors.

    However, real square matrices can have complex eigenvalues and eigenvectors. In this section, we introduce complex vector spaces, complex matrices, complex eigenvalues and complex eigenvectors.



Complex vector spaces

   

Definition

 [Complex Vector Space]

 

 

 

 

The set of vectors with complex components is denoted by

               .

 

If we define the vector addition and the scalar multiple of a vector in similar to those for , then is a vector space over and its dimension is equal to .

 

 

 

 

  

 If

          , , , ,


  then a vector in can be expressed as where ’s are complex numbers,

     and the set {, } is a basis for .  This basis is called the standard basis for .



 For a complex number , is called the conjugate of and  is called the modulus of .

    Furthermore, if we denote a complex number as , then and .

    For a complex vector , we define its conjugate as .


[Example] http://matrix.skku.ac.kr/RPG_English/9-VT-conjugate.html

그림입니다.
원본 그림의 이름: mem000013bc0003.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel


Inner product


 

Definition

 [Inner Product]

 

 

 

 

Let and be vectors in . Then

 

              

 

satisfies the following properties:

 

(1)

(2)

(3)

(4) in particular,

 

The inner product is called the Euclidean inner product for the vector space as a special case.

 

 

 

 



   

Definition

 

 

 

 

 

Let be vectors in .

      Then, using the Euclidean inner product , we can define the Euclidean norm of and

     the Euclidean distance between as the following:

 

(1)     .

(2)    .

 

 

 

 

  

 If , then we say that and are orthogonal to each other.


        그림입니다.
원본 그림의 이름: mem0000019c0006.gif
원본 그림의 크기: 가로 578pixel, 세로 167pixel     



For vectors , compute the Euclidean inner product and their Euclidean distance. 


   

       


   

                            


 

                                                                     

u=vector([2*I, 0, 1+3*I])  # I is the imaginary unit.

v=vector([2-I, 0, 1+3*I])

print v.hermitian_inner_product(u)  # Taking the conjugate for v

# < u, v > = v.hermitian_inner_product(u)

print (u-v).norm()

                                                                   

4*I + 8

sqrt(13)                                                          ■




Complex Eigenvalues and Eigenvectors of Real

Matrices


Let be a real square matrix of order .

  Since eigenvalues of are zeros of its  characteristic  polynomial, eigenvalue can be a complex number.

    If we find eigenvector corresponding to a complex eigenvalue, it will a complex vector.

   

Theorem 

 8.7.1

If is a complex eigenvalue of an real matrix and is its corresponding eigenvector,

     then the complex conjugate of is also an eigenvalue of and is an eigenvector corresponding to .


 Since an eigenvector is a nonzero vector, and . Since and is real (i.e.,),

    

                      it follows that .




Eigenvalues of Real Symmetric Matrices

   

Theorem

 8.7.2

If is a real symmetric matrix, then all the eigenvalues of are real numbers.


 Let be an eigenvalue of , that is, there exists a nonzero vector such that .

      By multiplying both sides by on the left-hand side, we get .

        Hence . Since is a nonzero real number, we just need to show that is a real number. Note that


               .

Therefore, is a real number.                         



Show that the eigenvalues of are . In addition show that if , then can be decomposed into


                      ,


where is the angle between the -axis and the line passing through the origin and the point .


Since the characteristic equation of is , the eigenvalues of are .

       If , then , . Therefore,


       .      


                         그림입니다.
원본 그림의 이름: CLP000016f812d4.bmp
원본 그림의 크기: 가로 390pixel, 세로 393pixel




8.8 Hermitian, Unitary, Normal Matrices

 

 Lecture Movie : http://youtu.be/8_uNVj_OIAkhttp://youtu.be/GLGwj6tzd60 

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-13-sec-8-8.html

그림입니다.
원본 그림의 이름: mem000007b40007.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

We used to denote the set of all real matrices.

      In this section, we introduce to denote the set of all complex matrices. 

    Symmetric matrices and orthogonal matrices in can be generalized to be Hermitian matrices and unitary matrices in ,

    We shall further study the diagonalization of Hermitian and Unitary matrices.




Conjugate Transpose

   

Definition

 [Conjugate Transpose]

 

 

 

 

For a matrix, is defined by

 

                    .

 

The transpose of the complex conjugate of is called the conjugate transpose and

   

         is denoted by , that is, .

 

 

 

 


   

[Remark] 

 

 

 

 

 

● The Euclidean inner product in : ,

● If a matrix is real, then .

 

 

 

 

 




For matrices , , , their conjugate transposes are


     , , .             

Theorem

 8.8.1 [Properties of Conjugate Transpose]

For complex matrices and a complex number , the following hold:

 

 (1) .

 (2) .

 (3) .

 (4) .

 Proof of the above theorem is easy and left as exercises.



Hermitian Matrix

   

Definition

 [Hermitian Matrix]

 

 

 

 

If a complex square matrix satisfying , is called a Hermitian matrix.

 

 

 

 


In , and hence is not Hermitian. However, since , is Hermitian.                                         



Theorem

 8.8.2 [Properties of Hermitian Matrix]

Suppose is Hermitian. Then the following hold:

 

(1) For any vector , the product is a real number.

(2) Every eigenvalue of is a real number.

(3) Eigenvectors of corresponding to distinct eigenvalues are

   orthogonal to each other.

  Theorem 8.7.2 & http://people.math.gatech.edu/~meyer/MA6701/module11.pdf



Let . Since , is Hermitian.

    The characteristic equation of is and hence the eigenvalues of are 1, ,

      which confirms that all the eigenvalues of a Hermitian matrix are real numbers

    . Furthermore, it can be shown that the eigenvectors are  , , and , where

       , ,

corresponding to , , and respectively, are orthogonal to each other.                                   





Skew-Hermitian Matrices


  

Definition

 [Skew-Hermitian Matrix]

 

 

 

 

If a complex square matrix satisfies , then is called a skew-Hermitian matrix.

 

 

 

 


It can be verified that both matrices and   below are skew-Hermitian:

       and

                                                        

  



 Every matrix can be expressed as , where is Hermitian and is skew-Hermitian.

     In particular, since is Hermitian and is skew-Hermitian, every complex square matrix can be rewritten as


                          .



Unitary Matrices

Definition

 [Unitary Matrix]

 

 

 

 

If matrix satisfies , then is called a unitary matrix. If is unitary, then . In addition, if the th column vector of is denoted by , then

 

              .

 

Therefore, is a unitary matrix if and only if the columns of form an orthonormal set in .

 

 

 

 



Show that the following matrix is unitary:


              


 Since , the product

   . Hence

 is a unitary matrix. We can also show that

                    .

For example, .          



  

Theorem

 8.8.3 [Properties of a Unitary Matrix]

Suppose has the Euclidean inner product and is a unitary matrix. Then the following hold:

 

(1) For , , which implies .

(2) If is an eigenvalue of , then .

(3) Eigenvectors of corresponding to distinct eigenvalues are orthogonal to each other.


 The property of a unitary matrix shows that a unitary matrix is an isometry, preserving the norm.



Unitary Similarity and Unitarily Diagonalizable

Matrices



Definition

 [Unitary Similarity and Unitary Diagonalization]

 

 

 

 

For matrices , if there exists a unitary matrix such that ,

      then we say that and are unitarily similar to each other.

     Furthermore, if is unitarily similar to a diagonal matrix, then is called unitarily diagonalizable.

 

 

 

 



Let and . Then it can be checked that is a unitary matrix and .

         Therefore, is unitarily diagonalizable.            

  


 If is unitarily diagonalizable, then there exists a unitary matrix such that and

                            hence . Letting , we get,


     .


 This implies that each th column of the unitary matrix is a unit eigenvector of corresponding to the eigenvalue .



Find a unitary matrix diagonalizing matrix .


The eigenvalues of are and their corresponding eigenvectors are


         .


Letting  ,

 and , it follows that


   ,


where is a unitary matrix.                            




Schur’s Theorem


 Transforming a complex square matrix into an upper triangular matrix

  

Theorem

 8.8.4 [Schur’s Theorem]

A square matrix is unitarily similar to an upper triangular matrix whose main diagonal entries are the eigenvalues of .

     That is, there exists a unitary matrix and an upper triangular matrix such that

 

                , (),

 

where ’s are eigenvalues of .

 

   Let be the eigenvalues of . We prove this by mathematical induction.

        First, if  , then the statement holds because . We now assume that the statement is true for any square matrix of order less than or equal to .

 

 ① Let be an eigenvector corresponding to eigenvalue .


②By the Gram-Schmidt Orthonormalization, there exists an orthonormal basis for including  , say .


③ Since is orthonormal, the matrix is a unitary matrix.

   In addition, since , the first column of is . Hence is of the following form:

 

                       .,

 

  where . Since , the eigenvalues of are  .


④ By the induction hypothesis, there exists a unitary matrix such that

                      .

 

⑤ Letting ,

 

we get

    .

  Since is a unitary matrix, the result follows.              


[Lecture on this proof] http://youtu.be/lL0VdTStJDM


 Not every square matrix is unitarily diagonalizable. (see Chapter 10)




Normal matrix

   

Definition

 [Normal Matrix]

 

 

 

 

If matrix satisfies

                                ,

then is called a normal matrix.

 

 

 

 



It can be shown that the following matrices and are normal:

              



A Hermitian matrix satisfies and hence . This implies that any Hermitian matrix is normal.

     In addition, since a unitary matrix satisfies , it is a normal matrix.   



Equivalent Conditions for a Matrix to be Normal

  

Theorem

 8.8.5

For matrix , the following are equivalent:

 

(1) is unitarily diagonalizable.

(2) is a normal matrix.

(3) has orthonormal eigenvectors.



Let   and .

Show that is a normal matrix and the columns of are orthonormal eigenvectors of .

Since , and hence is a normal matrix.

Letting , we get

                   

                 .


Thus and are eigenvectors of . In addition, since and , and are orthonormal eigenvectors of .   



Show that is unitarily diagonalizable.


Note that matrix is Hermitian and its eigenvalues are .

An eigenvector corresponding to is . By normalizing it, we get .

     Similarly, we can get a unit eigenvector corresponding to .

Taking , we get .                                       



[Remark]

 

 

 

 

 

Although not every matrix is diagonalizable, using the Schur’s Theorem,

      we can obtain an upper triangular matrix (close to a diagonal matrix) similar to .

     The upper triangular matrix is called the Jordan canonical form of .

   The Jordan canonical form will be discussed in Chapter 10.

 

 

 

 

 





8.9 *Linear system of differential equations

 

 Lecture Movie : http://www.youtube.com/watch?v=c0y5DcNQ8gs  

 Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-1.html

그림입니다.
원본 그림의 이름: mem000007b40008.tmp
원본 그림의 크기: 가로 357pixel, 세로 357pixel

Many problems in science and engineering can be written as a mathematical problem of solving linear system of differential equations.

   In this section, we learn how to solve linear system of differential equations by using a matrix diagonalization.

    


Details can be found in the following websites:

       http://www.math.psu.edu/tseng/class/Math251/Notes-LinearSystems.pdf

http://matrix.skku.ac.kr/EM-Sage/E-Math-Chapter-5.html

http://matrix.skku.ac.kr/CLAMC/chap8/Page83.htm  

http://matrix.skku.ac.kr/CLAMC/chap8/Page84.htm 

http://matrix.skku.ac.kr/CLAMC/chap8/Page85.htm 

http://matrix.skku.ac.kr/CLAMC/chap8/Page86.htm 

http://matrix.skku.ac.kr/CLAMC/chap8/Page87.htm


1) http://langvillea.people.cofc.edu/DISSECTION-LAB/Emmie'sLSI-SVDModule/p5module.html