Introductory Mathematics for Artificial Intelligence

by Prof. Sang-Gu Lee

· Week 6. Matrix decompositions (LU, QR, SVD) http://matrix.skku.ac.kr/intro-math4ai/W6/

· 07 Week http://matrix.skku.ac.kr/intro-math4ai/W7/

· 08 Week http://matrix.skku.ac.kr/intro-math4ai/W8/

· Week 9. Gradient descent method http://matrix.skku.ac.kr/intro-math4ai/W9/

· Week 10 http://matrix.skku.ac.kr/intro-math4AI/W10/

· Week 11 http://matrix.skku.ac.kr/intro-math4AI/W11/

· Week 12 PCA http://matrix.skku.ac.kr/intro-math4AI/W11/

· Week 13 ANN과 Backpropagation http://matrix.skku.ac.kr/intro-math4AI/W13/

· Week 14 MNIST Data Set and PCA http://matrix.skku.ac.kr/intro-math4AI/W14/

A summary by DongNa (Chinese Student)

1) [Day 1] summary

2) [Day 1] Code-practice

3) [Day 2] Summary

4) [Day 2] Code-practice

5) [Day 3] Summary

6) [Day 3] Code-practice

7) [Day 4] Summary

8) [Day 4] Code-practice

9) [Day 5] Code-practice

10) [Day 5] Summary

11) [Day 6] Code-practice

12) [Day 6] Summary

1) [Day 1] Summary

(12) Least Square Solution

(13) The method of finding least square line

(14) Gram-Schmidt

(15) QR-decomposition

(16) 최소제곱문제를 QR 분해로 풀기

[Day 1] Code-practice

1).Vector Projection

page43image54452352

Source: Linear Algebra

Sang-Gu LEE with Jon-Lark KIM, In-jae KIM,Namyong LEE,

Ajit KUMAR,Phong VU,Victoria LANG,Jae Hwa LEE

Where p: the projection of y to x

W: the component of y orthogonal to x

2).Distance Between a Point and a Plane

page45image54410064

Source: Linear Algebra

Sang-Gu LEE with Jon-Lark KIM, In-jae KIM,Namyong LEE,

2) [Day 2] Summary

2) Eigenvalue & Eigenvector

3) Diagonalizable Matrix

4) SVD

5) Least square solution

6) Pseudo-inverse

3) [Day 2] Code-practice

4) [Day 3] Summary

1) Differentiable

2) Derivative

3) Maximum & Minimum

4) Integral

5) Partial Derivative

6) Chain Rule

7) Gradient & Hessian

8) Directional derivative

9) Fermat’s theorem on critical points

5) [Day 3] Code-practice

6) [Day 4] Summary

2) Taylor

· 1) Hessian Matrix

2) Double Integration

3) Jacobian Determinant

4) Maximum & Minimum

5) Quadratic form

6) The relationship between vector & determinant

7) Inverse Matrix

8) The method of finding least square line

9) Gram-Schmidt

10) QR-decomposition

11) 최소제곱문제를 QR 분해로 풀기

7) [Day 4] Code-practice

[Day 5] Code-practice

[Day 5] Summary

[Day 6] Code-practice

[Day 6] Summary

3) The properties of PMF & PDF

4) Permutation & combination

18)Expectation & Variance & Standard Deviation

5) Covariance Matrix

6) Gamma Distribution & Beta Distribution

7) Joint Density Function

12) Central Limit Theorem

13) Covariance & Correlation Coefficient

14) Normal Distribution & Standard Normal Distribution

· 15) Joint Probability Function & Marginal Probability Distribution

16) Expectation & Variance & Standard Deviation

<Project Proposal and Some result>

Name / e-mail (이름과 이-메일): Dong Na

Team Project (Tentative, just idea now):

[First One] Economics key points +Mathematics AI key points

There is more detailed information in the picture as shown below.

[Second one] Blockchian(Bitcoin) & Prabability

I have already presented first topic last time, so I want to represent another topic which would relate to bitcoin & probability.

The total frame of the report

The aim of Economics is an effort to find optimal resource allocation under resource constraint. One of the classifications of economics is that static economics and dynamic economics. In terms of static economics, I think the most important thing is optimization. More specifically, I will apply two specific examples. The first example is how to find profits maximization. The method is to use Critical Value & Hessian Matrix from Basic Mathematics for AI. And the second example is the method of finding Utility Maximization by using Stationary Point & Bordered Hessian from Basic Mathematics for AI. Moreover, in terms of Dynamic Economics, Firstly, I will apply some economics definitions which can be calculated via using Differentiation & Integration from Basic Mathematics for AI; Secondly, I will take a specific case for solving the exhaustible resource problem for the optimal extraction path through using Differentiation & Integration from Basic Mathematics for AI.

Static Economic

First Example for Profit Maximization

By using: Critical value + Hessian Matrix

To find critical point of the given function by using sage-code

To determine the positive sign or negative sign of Hessian Matrix, then we can determine whether the maximum or minimum of the given function at the critical point

Thus, in my opinion, in terms of the first example for Profit Maximization, we can also calculate this problem by using sage-code. However, I am not good at using sage-code, so it is very hard for me to calculate the solution in this way. As we can see, It’s a very complicated way to find the solution without using sage-code. Therefore, my next goal is to try my best to study sage-code.

Static Economic

The Second Example for Utility Maximization

By using: Stationary Point + Bordered Hessian

Total Conclusion for Maximum & Minimum

Dynamic Economic

Some Economic definitions by using Differentiation + Integration

Dynamic Economic

One specific Example for solving the exhaustible resource problem for the optimal extraction path by using Differentiation + Integration

Conclusion, these problems of economics can all be easily and quickly solved by using sage-code. Thus, I think I should study sage-code as soon as possible. Sage-code is a very useful thing in many areas.

The reason why we firstly to study linear algebra is that all of things in this world can be described by using vectors. Moreover, linear algebra applies a best way to solve system of linear equations by using determinant & many types of matrix. Especially Hessian determinant & Jacobian determinant & inverse matrix & diagonal matrix & orthogonal matrix. Moreover, Hessian Determinant & Jacobian determinant can also be used in differential and probability. Moreover, we can use them to solve n-dimension of dataset. There are two main ways to reduce the higher dimension including SVD & PCA.

PCA is a dimension reduction method, which means to reduce the rank of the covariance matrix by using SVD. More specifically, the process of dimension reduction is that to delete some relative unimportant eigenvectors, then use the rest of eigenvectors to create a reduced space (a smaller size of matrix). Although this method has to lost some eigenvectors, the relative important eigenvector will be preserved in the end in order to make sure total information of the dataset is as same as possible. Moreover, PCA method is more efficient than the linear regression method due to the fact that there is minimum distance from each data to the linear function. (Each data is orthogonal to the linear function). Thus, if we use the reduced size of matrix to analysis and application of such an amount of dataset, we will efficiently deal with data.

Moreover, I am very interested in deep neural network. Amazing! The algorithm of deep neural network is that in order to decrease the error between predictive value and correct value, which means to update the weight by using back propagation and gradient descent method.

Math4AI, Professor: Sang-Gu LEE

(Summary by Dong-Na)

1) [Final OK by SGLee] Finalized by 손재민,김미리[6주차] 결합밀도함수 및 주변밀도함수(정리와 예제 풀이 추가)

[My Comments]: Jacobian can be used in probability & One example

2)[Final OK by TA] Re-Finalized by 이범수[6주차] Finalized by 박건영, 이상구, 손재민 (연속확률분포에 대한 질문)

[My Comment] the reason why I chose this topic is that I was interested in the python code. Excellent logic & beautiful graph. What a smart person!

3)[Final OK by SGLee] Finalized by 홍정명,유민솔,박건우-[6주차]질문-이항분포를 정규근사할 수 있는 이유

[My Comments] When n is large enough, in order to easily calculate the binomial distribution, we can use the approximate solution of standard normal distribution. (pf & one example)

4)[Final OK by SGLee]Re-finalized ver.2 by류재헌,박정현,이상구 교수님,이지용,이혜연,권서영,박건영-PCA가 데이터 분석에서 갖는 의의

[My Comments] PCA is a dimension reduction method, which means to reduce the rank of the covariance matrix by using SVD. More specifically, the process of dimension reduction is that to delete some relative unimportant eigenvectors, then use the rest of eigenvectors to create a reduced space (a smaller size of matrix). Although this method has to lost some eigenvectors, the relative important eigenvector will be preserved in the end in order to make sure total information of the dataset is as same as possible. Moreover, PCA method is more efficient than the linear regression method due to the fact that there is minimum distance from each data to the linear function. (Each data is orthogonal to the linear function). Thus, if we use the reduced size of matrix to analysis and application of such an amount of dataset, we will efficiently deal with data.

[Source]https://cn.ourladylakes.org/978756-singular-value-decomposition-algorithm-KBVLJQ

[Source] https://www.cnblogs.com/pxzheng/p/12690150.html

5)[Final OK by TA]Finalized by 권서영,이상구 교수님(comment on last year’s PBL report/신뢰구간에 대한 고찰)

[My Comments]: An example of in terms of normal distribution

Source: https://www.simplypsychology.org/confidence-interval.html

6) [Final OK by SGLee] Finalized by 이범수 오혜준[6주차] 연속확률분포와 파이썬의 구현

[My Comments]: the python code is very interesting. I analyzed one of pictures of Normal Distribution PDF as shown below.

7) [Final OK by SGLee]Re-finalized by 류재헌,김은진,권서우,유민솔,이상구 교수님,이상원,박수연,권서영 – 중심극한정리의 질문과 실습

[My Comments]

Source: https://medium.com/analytics-vidhya/central-limit-theorem-and-machine-learning-part-1-af3b65dc9d32

8) [Final OK by SGLee][6주차]Normal Distribution 이론,실습([Final OK by SGLee]Finalized by 이지용,안은선:[6주차]지수분포,정규분포 요약 글 인용하셨습니다)

[My Comments]

9) [Final OK by SGLee][Final OK by TA][5주차]Re-Finalized by 정승민,박건영,정진웅,양지원,박정현[포아송분포 유도하기,실습,조건,코드추가]

[My Comments] One pf.

10) [Final OK by SGLee] Re-finalized by 류재헌,유민솔,이지용,이상구 교수님,정현목,김은진-결합분보 질문

[My Comments] Summary of how to find 결합확률분포

[열린문제 1] 다른 교재에서 찾은 몇 가지 다항함수의 개형을 그리시오.

[ Hint : plot ( -x^9 -5*x^4 -130*x^3 -53*x^2 +542*x +3 , (x, -10, 10)) ]

[열린문제 2] 그래프의 개형을 그리시오.

Plot(x*sin(1/x))

[열린문제 3] 앞에서 배운 함수들 중 학습한 함수의 합성함수를 만들고, 그래프를 그리시오.

[열린문제 4] 다음 학생들의 문제풀이를 참고하여 다양한 방정식의 (근사)해를 구하시오.

http://matrix.skku.ac.kr/math4ai/PBL-Record/ http://matrix.skku.ac.kr/2020-Math4AI-PBL/

http://matrix.skku.ac.kr/KOFAC/ 에서 실습하시오

[열린문제 5] 행렬에 대해 학습한 다른 교재의 행렬 연산을 시행해 보시오.

http://matrix.skku.ac.kr/KOFAC/ 에서 실습하시오

[열린문제 6] 인터넷이나 다른 교재에서 5차 (이상) 행렬을 찾아서, 전치행렬과 역행렬이 존재하는지를 확인하고, 존재하면 찾아보시오.

[열린문제 7] 거리 척도를 사용하여 유사도를 계산할 수 있는 데이터의 종류에는 어떤 것이 있는지 생각해보시오.

[열린문제 8] 위의 거리 척도로 유사도를 판단하기가 용이하지 않은 데이터의 경우에 유사도를 판단하는데 사용이 가능한 다른 척도는 무엇이 있을지 생각해보시오. (Hint: 방향이 같은 데이터/벡터들의 경우)

[열린문제 9] 두 개의 7차원 벡터(데이터) 사이의 거리(distance)를 직접 구하시오.

(Hint: (거리를 활용한) 데이터의 유사도 <실습실> 활용)

http://matrix.skku.ac.kr/math4AI-tools/distance_similarity/

[열린문제 10] 어떤 데이터들이 코사인 유사도를 사용하여 분석 가능할지 생각해보시오.

[열린문제 11] 두 개의 5차원 데이터(벡터) 사이의 내적과 사잇각 를 구하시오. (Hint: (사잇각을 활용한) 데이터의 유사도 <실습실> 활용)

http://matrix.skku.ac.kr/math4AI-tools/cosine_similarity

[열린문제 12] 다른 교재의 선형 연립방적식의 해를 위의 명령어로 구하시오.

(Hint: http://matrix.skku.ac.kr/2018-album/LA-Sec-3-5-lab.html 실습 활용)

[열린문제 13] 주어진 선형연립방정식이 유일해를 갖는지, 무수히 많은 해를 갖는지, 해가 존재하지 않는지를 판단하는 것은 첨가행렬 의 RREE를 구하여 이것만 자세히 보면 바로 판단이 가능한 이유를 설명하시오.

1) Type 1

2) Tyope 2

3) Type 3

열린문제 14] 앞서 구한 방법으로 평면의 6개의 점에 (best fit 하는) 3차의 최소제곱곡선 을 구할 수 있음에 대하여 토론하시오.

[열린문제 15] 예제4와 같은 방법으로 다른 교재에서 찾은 선형연립방정식의 최소제곱해를 구하시오. (Hint: http://matrix.skku.ac.kr/2018-album/LS-QR-decom.html 활용)

[열린문제 16] 예제 6과 같은 방법으로 다른 교재에서 찾은 행렬의 특잇값 분해(SVD)를 구하시오.

[과제 2]

[열린문제 1] 다른 교재에서 찾은 (연속) 미분가능한 함수의 3계 도함수(3rd derivative)를 구하시오.

[열린문제 2] 주어진 구간에서 두 번 미분가능한 함수를 골라서 그 함수의 극댓값, 극솟값 및 그 구간에서의 최댓값, 최솟값을 찾아보시오.

[열린문제 3] 함수 의 최솟값을 구하시오. 단 , , 으로 한다.

I modified this solution.

The new solution shown as below.