4.Hessian matrix

Recall the Calculus

1) One varialbe(변수 1개)

In calculus, the second derivative test(이계도 함수 판정법) is a criterion for determining whether a given critical point(임계점) of a real function of one variable is a local maximum or a local minimum using the value of the second derivative at the point.

The test states: if the function f is twice differentiable at a critical point x (i.e. f'(x) = 0), then:

• If then has a local maximum(극대) at .
• If then has a local minimum(극소) at .
• If , the test is inconclusive.

2) Two variable(변수 2개)

Suppose that  is a critical point of  and that  has continuous second-order parital derivatives in some circular centered at

Define D(xy) to be the determinant

,

Then the second partial derivative test asserts the following:

1. If    and   then  is a local minimum(극소)of f.
2. If   and   then is a local maximum(극대) of f.
3. If   then  is a saddle point(안장점) of f.
4. If  then the second derivative test is inconclusive, and the point (ab) could be any of a minimum, maximum or saddle point.

Example) Locate the maxima, minima, and saddle points of the functions.

. 3) In 3 more variable,

How can we determine that those critical points are local minium or local maximum or saddle point?

We can think the matrix, , called  as the Hessian matrix of .

Define  to be the determinant .

Suppose that  is a critical point of  and that  has continuous second-order parital derivatives in some circular centered at

Then, we can rewrite the Second Derivative Test(이계도 함수 판정법) as following:

1.  has a local minimum at  if  is positive definite.

2.  has a local maximum at  if  is negative definite.

3.  has a saddle point at  if  is indefinite.

4. The test is inconclusive otherwise.

Definition4

Let   be symmetric matrix and let . Then :

• Positive definite(양의 정부호) if    for all nonzero .
x
  x  

• Positive semidefinite(양의 준정부호) if   for all .
• Negative definite(음의 정부호) if  for all nonzero .
• Negative semidefinite(음의 준정부호) if    for all .
• Indefinite(부정부호) if   takes both positive and negative values.

Theorem3

Let  is a symmetric matrix, then:

•   is positive definite if and only if all eigenvalues of  are positive.
•   is negative definite if and only if all eigenvalues of  are negative.
•   is indefinite if and only if  has at least one positive eigenvalue and at least one negative eigenvalue.

Example) For   , determine the natrue of critical point .

Sol)

The matrix  of quadratic form is    and its eigenvalues are  and

다음은 이변수함수에 관한 local extrema를 보여주는 interact입니다.(단, 임계점이 의 형태면 구해지지 않음)

In general(일반적으로),

Given the real-valued function if all second partial derivatives of  exist and are continuous over the domain of the function, then the Hessian matrix of  is where x = (x1x2, ..., xn) and  is the differentiation operator with respect to the th argument. Thus The following test can be applied at any critical point (ab, ...) for the Hessian matrix

1. If the Hessian is positive definite (equivalently, has all eigenvalues positive) at (ab, ...), then f attains a local minimum(극소) at (ab, ...).
2. If the Hessian is negative definite (equivalently, has all eigenvalues negative) at (ab, ...), then f attains a local maximum(극대) at (ab, ...).
3. If the Hessian has both positive and negative eigenvalues then (ab, ...) is a saddle point(안장점) for f   (and in fact this is true even if (ab, ...) is degenerate)

Remark

The mixed derivatives(혼합 도함수) of f are the entries off the main diagonal in the Hessian. Assuming that they are continuous, the order of differentiation does not matter (Clairaut's theorem). For example, :If the second derivatives of f are all continuous in a neighborhood D, then the Hessian of f is a symmetric matrix(대칭행렬) throughout D

Example) For the function   , find the critical points and determine whether the critical points are local maximum , local minimum or saddle point.

Sol)

The Hessian matrix of  is   .

At  , the eigenvalues of Hessian matrix are positive and negative values.

At  ,  the eigenvalues of Hessian matrix are positive and negative values.

Example) Locate the maxima, minima, and saddle points of the functions. 다음은 삼변수함수에 관한 local extrema를 보여주는 interact입니다.(단, 임계점이 의 형태면 구해지지 않음)

Theorem4

The following are equivalent for a real symmetrix matrix :

•   is positive definite
•  All the eigenvalues of  are positive
•  All the leading pricipal submatrix 's have positve determinants
•  All the pivots (without row interchanges) are positive
•  There exists a nonsigular matrix  such that
•  There is an symmetric positive definite matrix  such that

There is sample of proof in case that  is positive definite (Principal axes Theorem(2013.12.02), Cholesky Theorm(2013.12.02))  The following are equivalent for a real symmetrix matrix :

•   is negaitive definite
•  All the eigenvalues of  are negaitive
•  The determiant  for , i.e,
•  The quadratic forms associated to all  are negative definite
•  All the pivots (without row interchanges) are negaitive
• We can prove other things using above the proof of positive definite.

The following are equivalent for a real symmetrix matrix :

•   is positive semidefinite
•  All the eigenvalues of  are nonnegative
•  All the leading pricipal submatrix 's have nonnegative determinants
•  All the pivots (without row interchanges) are nonnegative
•  There exists a matrix , possible singular, such that
• The followibg are equivalent for a real symmetrix matrix :

•   is negative semidefinite
•  All the eigenvalues of  are nonpositive
•  All the leading pricipal submatrix 's have nonpositive determinants
•  All the pivots (without row interchanges) are nonpositive
•  There exists a matrix , possible singular, such that

Reference)

Recall the calclus-one variable- Second derivative test, wikipedia, http://goo.gl/W11Os,2013.11.26

- two variable& in 3 more variable- Second partial deribvative test, wikipedia, http://goo.gl/R0LQMS,2013.11.26

In General, Information of Hessian matrix- Hessian matrix, wikipedia, http://goo.gl/s24ns, 2013.11.26

Definition 4, Theorem3 , Theorem4 and examples- Linear Algebra, Kwak&Hong, p294~p297.

Back TO the HOME