PBL Report (개인성찰 노트)
2020년 도전학기 Midterm PBL Report (Action Learning)
Class: Basic Math for AI
(인공지능을 위한 기초수학)
Professor: SangGu LEE
Due day: 2020/08/10 (Monday, 11:00 am)
Name:
Yuhai Oleksandr (유가이 올렉산드르)
Major:
BioMechatronic Engineering
Student Number: 2018
(1) State more than 10 Math Definitions and concepts what
you learned in Part 0, 1 and 2.
Part 0 (인공지능 개론) 
Briefly reviewed the basic functions of Python,
Sage and R, Basic concept of Power Method(거듭제곱법), Basic concept of MNIST 데이터셋을 활용한 손 글씨 숫자 인식(패턴인식), Briefly
familiarized myself with the principle of work of AlphaGo… 
Part 1 (인공지능 행렬과 데이터분석) 
벡터, 내적, 벡터의 노름, 코시슈바르치 부등식, 정사영(projection), 선형연립방정식(system of linear equations), 첨가행렬(augmented
matrix), 기본행 연산(ERO), 행 사다리꼴(REF), Row
Reduced Echelon Form, Gauss 소거법과 GaussJordan 소거법, 행렬과 행렬식, vector sum,
scalar multiplication, 행렬의 곱(product), 영행렬(zero matrix), trace, 역행렬, 가역행렬의 성질, 대각행렬, 스칼라행렬, 대칭행렬, 치환(permutation), 수반행렬(adjugate, adjunct), 일차독립, 일차종속, Subspace, 기저, 차원, nullity, column space and row space, 계수(rank), RankNullity
정리, 최소재곱해(least square solution), Normal equation, Orthonormal basis,
GramSchmidt 정규직교화 과정, QR 분해, 선형변환(Linear Transformation), 표준행렬, 핵(kernel)과 치역(range), 동형사상(isomorphism), 고윳값, 고유벡터, 대각화(Diagonalization), 닮음 행렬(similar matrix), 직교대각화(orthogonally diagonalizing), 고윳값분해(eigendecomposition), SVD(특이값 분해), 일반화된 역행렬(Pseudoinverse, MoorePenrose Generalized Inverse), 이차형식(quadratic
form), 주축정리(Principal axis theorem)… 
Part 2 (다변수 미적분학과 최적화) 
함수, 극한(limit), 도함수(derivative)와 미분(differentiation), 뉴턴 방법(Newton’s Method
with gradient), 적분(Integral), 리만 합(Riemann sum), 외적(Cross Product), 벡터 함수, 편도함수(Partial Derivative)와 그래디언트(gradient), 연쇄법칙(Chain
Rule), 방향도함수(directional
derivative), Hessian, Taylor 정리, 함수의 극대(Local Maximum) and 극소(Local Minimum), Fermat’s theorem on critical
points, 임계점(critical point), 안정점(saddle point), Gradient
Descent Algorithm(경사기울기 하강법), 중적분(double
integral, multiple integral), Fubini의 정리, Polar
Coordinates에서의 2중적분, Jacobian…

Part 0 (인공지능 개론)
1.
Python is an interpreted,
highlevel, generalpurpose programming language. Python has become highly
popular due to its abundance of libraries. Each Python library is
applicationoriented that was developed to address problems. Mathematical
operations are most preferably carried out in Python due to its focus on
utility and minimal programming jargon. There are several libraries that can be
used to carry out mathematical operations with Python. Ex.: Math,
Numpy, SciPy, Scikitlearn etc.
2.
SageMath is a free opensource
mathematics software system licensed under the GPL. It builds on top of many
existing opensource packages: NumPy, SciPy, matplotlib, Sympy, Maxima, GAP,
FLINT, R and many more. Access their combined power through a common, Pythonbased
language or directly via interfaces or wrappers.
3.
R is a programming language for
statistical data processing and graphics, and is a free and open source
computing environment under the GNU Project.
4.
Power method is an eigenvalue
algorithm: given a diagonalizable matrix A, the algorithm will produce a number
λ, which is the greatest (in absolute value) eigenvalue of A, and a nonzero
vector v, which is a corresponding eigenvector of λ, that is Av = λv.
5.
MNIST(Modified National Institute
of Standards and Technology) database is a voluminous database of handwritten
numbers. The database is a standard proposed by the US National Institute of
Standards and Technology for the purpose of calibrating and comparing image
recognition methods using machine learning primarily based on neural networks.
6. AlphaGo
is a "Go game" program developed by Google DeepMind in 2015. AlphaGo
combines a technique that provided a major breakthrough in the power of
"Go game" software in 20072012 (MonteCarlo tree search) and
recent advances in machine learning, namely deep learning with multilevel
neural networks.
Part 1 (인공지능 행렬과 데이터분석)
1.
Vector is an object that has both
a magnitude and a direction. Geometrically, we can picture a
vector as a directed line segment, whose length is the magnitude of the vector
and with an arrow indicating the direction. The direction of the vector is from
its tail to its head.
2.
Vector projection of a vector a
on (or onto) a nonzero vector b is the orthogonal projection of a
onto a straight line parallel to b. It is a vector parallel to b,
defined as: a_{1} = a_{1}, where a_{1} is a scalar,
called the scalar projection of a onto b, and is the
unit vector in the direction of b. Note that, scalar projection
is defined as:
a_{1} = a cos θ = a · = a ·
where · denotes a dot product,
a is the length of a, and θ is
the angle between a and b.
Note that, both the projection
a_{1} and rejection a_{2} of a
vector a are vectors, and their sum is equal to a,
which implies that the rejection is given by: a_{2} = a – a_{1}.
3.
기본행연산: 기본행연산법에는 3개의 방법이 있다.
·
두 행을 교환 (R_{i} ó R_{j}).
·
한 행에 0이 아닌 상수를 곱함 (kR_{i} ó R_{i}, where k ≠ 0).
·
한 행의 배수를 다른 행에 더함 (R_{i} + kR_{j}
=> R_{i}, where i ≠ j).
4.
Gauss
소거법: 선형연립방정식의 첨가행렬을 REF로 변형하여 푸는 방법이다.
5.
GaussJordan
소거법: 선형연립방정식의 첨가행렬을 RREF로 변형하여 푸는 방법이다.
6.
행렬의 곱셈(세종대왕의 기역 법칙):
두 행렬 A = [a_{ij}]_{mxp},
B = [b_{ij}]_{pxn}에 대하여 A와 B의 곱(product)
AB를 다음과 같이 정의한다.
AB = [c_{ij}]_{mxn}
여기서,
c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j}
+ ··· + a_{ip}b_{pj} = (1 ≤ i ≤
m, 1 ≤ j ≤ n)
7.
가역 행렬(invertible
matrix)은 그와 곱한 결과가 단위 행렬인 행렬을 갖는 행렬이다. 이를 그 행렬의 역행렬(inverse matrix)이라고 한다.
8.
대칭 행렬(symmetric
matrix)은 전치 행렬(transposed matrix)이 스스로와 같은 행렬이다 (A = A^{T}).
9.
치환행렬(permutation matrix) 은 순서가 부여된 임의의 행렬을 의도된 다른 순서로 뒤섞는 연산 행렬이다.
일반적으로 치환행렬은 단위행렬로부터 얻을수있는 이진 행렬이다. 반대각행렬은 치환행렬의 특수한 경우이다.
10. 계수퇴화차수 정리(ranknullity
theorem)는 행렬의 상과 핵의 차원의 관계에 대한 정리이다.
11. QR 분해(QR
decomposition)는 실수 행렬을 직교 행렬과 상삼각 행렬의 곱으로 나타내는 행렬 분해이다. 그람슈미트 과정이나 하우스홀더 행렬이나 기븐스 회전을 통해 얻을 수 있으며, 선형 최소 제곱법이나 QR 알고리즘에서 쓰인다.
12. 고유벡터(eigenvector)는 그 선형 변환이 일어난 후에도 방향이 변하지 않는,
영벡터가 아닌 벡터이다.
13. 고윳값(eigenvalue)는 고유벡터의 길이가 변하는 배수를 선형 변환의 그 고유 벡터에 대응하는 값이다.
14. 대각화 가능 행렬(diagonalizable
matrix)은 적절한 가역 행렬로의 켤레를 취하여 대각 행렬로 만들 수 있는 정사각 행렬이다.
15. 고유값 분해(eigen
decomposition)는 고유값 과 고유벡터로 부터 유도되는 고유값 행렬과 고유벡터 행렬에 의해 분해될수있는 행렬의 표현이다.
16. 특이값 분해(Singular
Value Decomposition)는 행렬을 특정한 구조로 분해하는 방식으로, 신호 처리와 통계학 등의 분야 에서 자주 사용된다.
17. Quadratic
form is a function on a vector space, specified by a homogeneous polynomial
of the second degree in the coordinates of the vector.
18. Principal
axis is a certain line in a Euclidean space associated with an ellipsoid or
hyperboloid, generalizing the major and minor axes of an ellipse or hyperbola.
The principal axis theorem states that the principal axes are
perpendicular, and gives a constructive procedure for finding them.
Part 2 (다변수 미적분학과 최적화)
1.
Function(함수) is a binary relation over two sets that associates
to every element of the first set exactly one element of the second set.
2.
Riemann sum is a certain kind of
approximation of an integral by a finite sum. One very common application is
approximating the area of functions or lines on a graph, but also the length of
curves and other approximations.
3.
Partial derivative of a function
of several variables is its derivative with respect to one of those variables,
with the others held constant.
4.
기울기(gradient)란 벡터 미적분학에서 스칼라장의 최대의 증가율을 나타내는 벡터장을 뜻한다. 기울기를 나타내는 벡터장을 화살표로 표시할 때 화살표의 방향은 증가율이 최대가 되는 방향이며, 화살표의 크기는 증가율이 최대일 때의 증가율의 크기를 나타낸다.
5.
Fermat's theorem is a method to
find local maxima and minima of differentiable functions on open sets by
showing that every local extremum of the function is a stationary point (the
function derivative is zero at that point).
6.
경사 하강법(Gradient
descent)은 1차 근삿값 발견용 최적화 알고리즘이다. 기본 개념은 함수의 기울기(경사)를 구하여 기울기가 낮은 쪽으로 계속 이동시켜서 극값에 이를 때까지 반복시키는 것이다.
7.
Multiple integral is a definite
integral of a function of more than one real variable, for instance, f(x,
y) or f(x, y, z).
8. 야코비 행렬(
Jacobian matrix)은 다변수 벡터 함수의 도함수 행렬이다.
(2) State more than 5 things that you know/can/find
after you studied the first Part 1 and 2. (Please, note that, I will only indicate the most
important terms(in my opinion) that I remember the most.)
1. After
completing part 1, I can freely find the Inverse of the matrix.
Below I will give in my
opinion the fastest and simplest method for finding the inverse of a matrix. A method for
finding the inverse of a matrix using Linear Row Reduction.
Step1: Adjoint the identity matrix to the original matrix.
Step2:
Perform linear row reduction operations.
Step3:
Continue until you form the identity matrix (on the left side of the newly
augmented matrix).
Step 4:
Write out the inverse matrix.
In Sage,
let's say we have a square matrix A, to find its inverse we can use the
command: A.inverse ().
2. Also,
I can freely use GramSchmidt Process for orthonormalizing a set of
vectors in an inner product space. Imagine that we are given a basis {x_{1},
x_{2}, … ,x_{p}} for a nonzero subspace W
or R^{n}, so, we can define that,
v_{1} = x_{1}
v_{2} = x_{2} 
v_{3} = x_{3}  
⋮
v_{p} = x_{p}    ··· 
Then {v_{1},
… ,v_{p}} is an orthogonal basis for W. In addition,
note that,
Span {v_{1}, … ,v_{k}} =
Span{x_{1}, … ,x_{k}} for 1 ≤ k ≤ p
In Sage,
we can easily applicate GramSchmidt process using the command
(In the case of matrix A):
A.gram_schmidt().
3. I
consider Singular Value Decomposition(SVD) to be one of the key topics
of linear algebra, thanks to the professor and other students, I understood
this topic more deeply. Let's go over the basic steps for SVD (Again, let's
call our matrix A):
Step1:
Compute transpose of our matrix, A^{T}, and also compute A^{T}A.
Step2:
Determine the eigenvalues of A^{T}A and sort these in
descending order, in the absolute sense. Then, square roots these to obtain the
singular values of A.
Step3:
Construct diagonal matrix Σ by placing singular values in descending
order along its diagonal. Then, compute its inverse, Σ^{1}.
Step4:
Use the order eigenvalues from step 2 and compute the eigenvectors of A^{T}A.
Place these eigenvectors along the columns of V and compute its
transpose, V^{T}.
Step5:
Compute U as U = AVΣ^{1}. To compute the
proof, compute the full SVD using A = UΣV^{T}.
In Sage,
use the following code to calculate the SVD (an example is given for a 2x2
matrix, but you can use it for larger matrices, just slightly correcting the
code in some places):
A = matrix([Given Matrix A])
B = A.transpose()*A
eig = B.eigenvalues()
sv = [sqrt(i) for i in eig] #특이값 구하기
print(B.eigenvectors_right()) #(A^T)(A)의 고유벡터 구하기, A의 right singular vectors
G = matrix([Matrix formed by Eigenvectors])
Vh = matrix([1/G.row(j).norm()*G.row(j) for j in
range(0,2)]) #V의 전치행렬; Note: change range value based on matrix size
Vh = Vh.simplify() #V의 전치행렬 표현
print(Vh)
U = matrix([A*Vh.row(j)/sv[j] for j in
range(0,2)]).transpose()
print(U) # U의 열들을 A의 left singular vectors
S = diagonal_matrix(sv)
print(S)
print(U*S*Vh)
4. I
also fully understand and can use Newton's Method in optimization.
Suppose you need to find the root of a continuous, differentiable function f(x),
and you know the root you are looking for is near the point x = x_{0}.
Then Newton’s method tell us that a better approximation for the root is
x_{1} = x_{0} 
Repeat
this process as many times as necessary to get desired accuracy. In general,
for any xvalue x_{n}, the next value is
given by
x_{n+1} = x_{n} –
In Sage, you can use Newton's Method as follows:
#Use a loop
to compute iterations of NewtonRaphson's Method
x = var('x')
f(x) = Given
function
fp =
diff(f,x)
NewtonIteration(x)
= x  (f/fp)(x)
xn = Initial
guess;
print('x0 =',
xn);
for i in
range(10): #Number of interations
xn =
N(NewtonIteration(xn),digits=10);
print('x', i+1, '=', xn);
print("This
is the exact answer for comparison: ")
print(solve(f(x)==0,
x))
5. After studying part two, I fully understood the topic
of multiple integrals. In particular, I understood well the meaning of Fubini's
Theorem. Fubini's Theorem is a result that gives conditions under which it
is possible to compute a double integral by using an iterated integral. One may
switch the order of integration if the double integral yields a finite answer
when the integrand is replaced by its absolute value.
As a
consequence, it allows the order of integration to be changed in certain
iterated integrals. Fubini's theorem implies that two iterated integrals are
equal to the corresponding double integral across its integrands.
In Sage,
you can calculate multiple integrals using the following code (I will give an
example code for calculating a double integral, but it can also be used for a
triple integral or more):
f(x, y) = Given function
#in this example, I’ll integrate in the order dy dx
f(x, y).integrate(y, lower limit, upper
limit).inegrate(x, lower limit, upper limit)
6. One
of the most important topics in Part 2 is Gradient Descent
Algorithm. Following are the main steps for using
GradientDescent Algorithm.
Step1: Sets the initial
guess and precision of the calculation , ε.
Step2:
Then, calculate = – λ^{[j]} ∇F(), where λ^{[j]}
= argmin_{λ}F(  λ^{[j]} ∇F())
Step3:
Finally, check the stop condition:
·
If,
  > ε; F() – F() > ε; or ∇F() > ε (choose one of the conditions), then j = j + 1
and we need to go to step 2
again.
· Otherwise, = and stop.
Example Sage code from the textbook for GradientDescent Algorithm
problem:
# initializing
A = diagonal_matrix(RR, [20, 10, 2, 1])
b = vector(RR, [1, 1, 1, 1]) # objective function
x0 = vector(RR, [0, 0, 0, 0]) # initial guess
g0 = b # initial gradient
r = [] # 그래프를 그리기 위한 용도
# main iteration
for i in range(0, 200):
gn = g0.norm()
r.append((i, gn))
if gn < 10^(8):
print("Stationary point! Algorithm terminated!")
break
w = A*g0
a = g0.inner_product(g0)/(g0.inner_product(w)) # stepsize
x1 = x0a*g0
g1 = A*x1b
x0 = x1;
g0 = g1
show(line2d(r) + point(r, color = 'red')) # gradient 의 norm을 그래프로 그림
References:
[1] 이상구, 이재화., (2019), “인공지능을 위한 기초수학”, BigBook, Part 0,
Part I and Part II
[2] David
C. Lay, Steven R. Lay, Judi J. McDonald., (2016),“Linear Algebra and its
applications”, Pearson, pp. 63~78
[3]
James Stewart., (2016), “Calculus – Early Transcendentals”, Cengage Learning,
pp. 10~270, 276~291; 345~348;
[4] Wikipedia.,
(2020), “Definition of some concepts from Parts 0, 1 and 2 above”, Wikimedia
Foundation
· All the above pictures were used in accordance with the CC (Creative Commons) license.
This means that I can freely use, share
and adapt the content.
(3) State your
meaningful Comment/Answer/Discussions in Discussion/QnA. (Note: I will cite
just a few of my responses, comments that I find really meaningful and
helpful.)
1. Final by 천가영, 유가이 and SGLee [HW] 벡터에 관한 질문 by 천가영 < 답변 by 유가이
1. 벡터는 크기와 방향을 갖는 유향선분  2차원, 3차원 공간의 벡터는 화살표로 표현 가능
> Q1. 차원 벡터도 존재하나요?
답:
존재합니다.
만약 존재한다면
1차원 벡터는 화살표로 표현 불가능한가요?
답:
화살표로 표현하지 못하는 벡터들도 존재합니다)
아니면
1차원은 공간으로 간주되지 않아서 그런 것인가요?
답:
아닙니다.
1차원 벡터 공간도 존재합니다.
2.
(R^n 안의)
벡터
X의 노름은 원점에서 점
P(x1, x2, ... , xn)에 이르는 거리로 정의됨을 의미한다.
>Q2 아래의 그림과 같이 시점이 원점이 아닌 벡터는 두 벡터의 합으로 밖에 표현할 수 없나요?
답: 아닙니다. 시점이 원점이 아닌 벡터는 원점으로 옮겨놓고 norm 노름을 구하시면 됩니다)
Remark by 유가이 올렉산드르:
in answering this question, I tried to explain the concept of a vector in as much detail as possible. I also explained the concept of a
vector in different spaces and gave several examples for a deeper
understanding. Also thanks to the Professor's help (additional
material and correction of minor inaccuracies), I hope that this answer cleared
up all the student's doubts about the vector in space.
2. [Final OK by SGLee] Finalized by 한수현, 유가이 올렉산드르, 박지수] [HW해공간을 구하는 법, Kernel(Null Space), Right Null Space, Left Null
Space
Remark by 유가이 올렉산드르: Adding an
additional answer and finalizing this post, I decided to tell more about
Kernel(Null Space). I also explained the difference between right kernel and
left kernel and used python for visualization. In the end I showed a slight
difference when calculating null space in saga and python (using sympy library).
3. [Final OK by TA] [Final] Q by 김진웅 A by 유가이올렉산드르, 이상구 F by 김진웅 [HW 질문] 서로 같은 고유값에 대응하는 고유벡터들 차원 (JCF)
어떤 행렬 A 와 B 가 주어졌을 때 ...
서로 같은 (중근의) 고유값에 대응하는 (일차독립인)
고유벡터들이 이루는
부분공간 (eigenspace) 의 차원이 다를 때가 있는데 ...
어떻게 구분하여 알 수 있나요?
A 의
닮은 대각선행렬
:
B 의
닮은 대각선행렬 :
라는 의미는?
무엇인가요?
Here is my answer:
유가이올렉산드르(2018****33)7월 21일 오전 2:58
Let's
try to analyze the given example.
The first matrix "D = ([[1, 0], [0,
1]])" has two eigenvalues (1 and 1) but they are obviously not distinct.
Since A is the identity matrix, Av=v for any vector v, i.e. any vector is an
eigenvector of A. We can thus find two linearly independent eigenvectors (say
<2,1> and <3,2>) one for each eigenvalue.
But,
the second matrix "D = ([[1, 0], [1, 1]])" also has nondistinct
eigenvalues of 1 and 1. All eigenvalues are solutions of (AI)v=0 and are thus
of the form <t,0>. Hence, in this case there do not exist two linearly
independent eigenvectors for the two eigenvalues 1 and 1 since <t,0> and
<s,0> are not linearly independent for any values of s and t.
Remark
1: Every eigenvalue with multiplicity = n will be associated with n different
(as in linearly independent) eigenvalues.
Multiplicity
is how many "times" it shows up as an eigenvalue. It is like when you
find only one solution to a second degree equation, which always has two roots.
This solution has a multiplicity = 2.
Remark
2: If a matrix has more than one eigenvector the associated eigenvalues can be
different for the different eigenvectors.
Geometrically,
the action of a matrix on one of its eigenvectors causes the vector to stretch
(or shrink) and/or reverse direction.
4. [Final OK by SGLee] Finalized by 나종진,
답변 by 유가이올렉산드르
, 직교대각화가능할 필요충분조건(교재 p.125) 질문 by 나종진
Remark by 유가이 올렉산드르:
The student had minor problems with Sage code for Orthogonal Diagonalization, so I slightly corrected the code here.
My answer:
유가이올렉산드르(2018****33)7월 23일 오후 7:55
12번째 라인에서 C.row(i) => C.column(i)로 수정하시면 될 것 같습니다.
A=matrix([[0,3,3], [3,0,3], [3,3,0]])
print(A.eigenvectors_right())
print
x1=vector([1,0,1])
x2=vector([0,1,1])
x3=vector([1,1,1])
B=matrix([x1, x2])
[G, mu] = B.gram_schmidt()
print(G)
print y1=vector([1,0,1])
y2=vector([1/2, 1, 1/2])
C = column_matrix([y1,y2,x3])
P = column_matrix([C.column(i) /
C.column(i).norm() for i in range(0, 3)])
print("P =")
print(P)
print
print(P.transpose()*A*P)
유가이올렉산드르(2018****33)7월 23일 오후 8:26
코드를 실행해봤는데 예제의 답과 같은 답이 나옵니다.
5. 답 by 유가이 올렉산드르, 이상구 교수님, 질문 by 한수현, Newton's Method에 대한 질문
두 설명(with and without gradient) 모두 Newton's method라고 하는 것을 보면, 각각이 결국 같은 방식을 이용하여 원하는 근사값을 구한다는 의미로 보입니다. 하지만 저는 저 두 내용이 완전히 관련 없는 독립된 내용으로 느껴집니다. 그래서 저 두 내용의 상관관계를 설명해주시면 좋겠습니다.
유가이올렉산드르(2018****33)8월 9일 오전 5:21
I think in both cases we are talking about the
same iterative method (Newton's method). The difference is played by the
dimension in which we work. As you know, in calculus, Newton's method is an
iterative method for finding the roots of a differentiable function f, which
are solutions to the equation f(x) = 0. In optimization, Newton's method is
applied to the derivative f′ of a twicedifferentiable function f to find the
roots of the derivative (solutions to f′(x) = 0), also known as the stationary
points of f. These solutions may be minima, maxima, or saddle points. In
relatively Low dimensions we use classical Newton's method, but in Higher
Dimensions we can also use Newton's method, all we need to do is replace the
derivative with a gradient. also, please note that the geometric meaning in
both cases remains the same. Also often Newton's method(with gradient) is modified
to include a small step size 0 < gamma <= 1 instead of gamma = 1(pay
attention to the classic formula, there the gamma is 1). Below I tried (taking
into account the possibilities of the comments section) to give the basic
formula that the Newtonian method represents (with and without a gradient).
x_{k+1} = x_{k}  f'(x_{k}) / f''(x_{k}) and x_{k+1} = x_{k} 
gamma[f''(x_{k})] / f'(x_{k}) The method is the same, just the area of
application is slightly different. Newton's method with gradient is often done
to ensure that the Wolfe conditions are satisfied at each step of the method.
For step sizes other than 1, the method is often referred to as the relaxed or
damped Newton's method. The geometric interpretation of Newton's method is that
at each iteration, it amounts to the fitting of a paraboloid to the surface of
f(x) at the trial value x_ {k}, having the same slopes and curvature as the
surface at that point, and then proceeding to the maximum or minimum of that
paraboloid (in higher dimensions, this may also be a saddle point). The central
problem of optimization is minimization of functions, and Newton's Method copes
with it relatively quickly. Hope this was helpful. Please correct me if I was
wrong somewhere. [Reference: Read more on https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization]
이상구(LEE SANGGU)8월 9일 오전 6:57
Good. 같은 뉴턴 알고리즘 입니다. 차원만 인공지능 에서 사용하는 고차원으로 일반화 한것입니다.
(4) Below, I have consistently indicated
all my QnA activity.
[1.1] [1st HW] 간단한 자기소개와 수강동기, 유가이올렉산드르
[1.2] [Final OK by SGLee] [수정본] 추가답변 및 Refinalized by 유가이 올렉산드르, Finalized by 김범준 [1주차HW질문] 일차독립
[1.3] [Final OK by TA] Final [HW (1주차 배운 내용 요약 1] 나종진 + 유가이 : 다항함수의 개형을 그리기 + [HW] 질문 Q4. 교재 p.110(PDF p.118)와 답 by 유가이올렉산드르, 나종진, 이상구교수님
[1.4] [Final OK by TA and SGLee] Final by 천가영, 유가이 and SGLee [HW] 벡터에 관한 질문 by 천가영 < 답변 by 유가이
[1.5] Finalized by 유가이 올렉산드르 21강에 대해 질문 by 이상현, 답변 by 이상구 교수님
[1.6] [Final OK by SGLee] Finalized by 한수현, 유가이 올렉산드르, 박지수] [HW해공간을 구하는 법, f Kernel(Null Space), Right Null Space, Left Null
Space
[1.7] [Final OK by TA] [Final] Q by 김진웅 A by 유가이올렉산드르, 이상구 F by 김진웅 [HW 질문] 서로 같은 고유값에 대응하는 고유벡터들 차원 (JCF)
[1.8] [Final OK by TA] [HW] 열공간, 해공간, 영공간 사이의 관계, 질문 by 김진웅, 답변 by 장환승, 추가답변 및 Finalized by 유가이 올렉산드르
[1.9] [Final OK by SGLee] [HW] 1주차 강의 내용 요약 및 정리 + Sage과 Python 실습(비교), 내용 요약 및 정리 by 박진형, 코딩실습 및 Finalized by 유가이 올렉산드르
[2.1] [HW2] 선형변환 (Linear Transformation) 강의 요약 및 정리 + 연습문제 풀기
[2.2] [Final OK by SGLee] Finalized by 나종진, 답변 by 유가이올렉산드르 , 직교대각화가능할 필요충분조건(교재 p.125) 질문 by 나종진
[2.3] [HW2] 고윳값(eigenvalue), 고유벡터(eigenvector), 대각화(Diagonalization) 강의 요약 및 정리 + 연습문제 풀기
[2.4] [Final ok by SGLee] [HW2] 특이값 분해(Singular Value Decomposition, SVD) 강의 요약 및 정리 + 연습문제 풀기
[2.5] [Final OK by TA] 1주차 강의 내용 요약 및 정리 박진형, 코딩실습 및 Finalized by 유가이 올렉산드르
[3.1] [HW3] 함수(function), 극한(limit), 도함수(derivative)와 미분(differentiation) 강의 요약 및 정리 + 연습문제 풀기 + 코딩 실습
[3.2] [HW3] 미분의 응용 (Applications), 뉴턴랩슨법 강의 요약 및 정리 + 근사 찾기 방법 비교 + 연습문제 풀기 + 코딩 실습
[3.3] [HW3] 적분 (Integral) 강의 요약 및 정리 + 연습문제 풀기 + 코딩 실습
[3.4] [Final OK by SGLee] [HW3] Finalized by 유가이 올렉산드르, 답변 by 이상구 교수님, Solved by 이상현  뉴턴 방법 증명
[4.1] [Final OK by SGLee] [HW 4] 연습문제 풀기 및 ReFinalized by 유가이 올렉산드르, AI 을 위한 기초수학 16강 <미분적분학> [Gradient descent Algorithm] 개념 복습 및 정리 by 손원식, 추가 comment by 임성규, 이상구 교수님, 천가영
[4.2] Newton's Method에 대한 질문, 질문 by 한수현, 답 by 유가이 올렉산드르, 이상구 교수님
[4.3] Final OK by SGLee. [HW 4] 연습문제 풀기 및 ReFinalized by 유가이 올렉산드르, Finalized by 손원식, AI 을 위한 기초수학 17강 <미분적분학> [중적분 (double integral, multiple integral)] 개념 복습 및 정리
[4.4] 경사하강법 실습(Newton's Method), 질문 by 한수현, 답 by 유가이 올렉산드르, 이상구 교수님, 김정한, 박진형
Ch. 2: Participation Part (참여부분, 정량) 자기 평가와 본인의 Project (Term paper) 제안서 등에 대해 아래를 채우시오. (20점)
Fill
in the below for your selfassessment and your project/term paper. (20 points/100)
A. (Quantity 10pts)
Briefly describe your contributions through Q&A for yourself and fellow
students in our "DM" classes!
(1) Quantity: Check your participation numbers in
Q&A <Refer your name in Q&A directly>: Each
week (Saturday to Friday)
▪ Week
1: 9
Week 2: 5
Week 3: 4 Week 4: 4 (Total = entries) 22
Total number of sessions
(Q: 7 , A: 15 )
■ Number
of online attendances: ( 23 ) / 23 (14^{ }week)
Online WebEx OH attendance:
( 3 ) / 3 (14^{ }week)
Absences: 0
(2) 다음 밑줄 친 곳에 들어갈 내용을 고르시오.
나는 벡터, 내적, 벡터의 노름, 코시슈바르치 부등식, 정사영(projection), 선형연립방정식(system of linear
equations), 첨가행렬(augmented matrix), 기본행 연산(ERO), 행 사다리꼴(REF), Row Reduced
Echelon Form, Gauss 소거법과 GaussJordan 소거법, 행렬과 행렬식, vector sum,
scalar multiplication, 행렬의 곱(product), 영행렬(zero matrix), 역행렬, 대각행렬, 스칼라행렬, 대칭행렬, 치환(permutation), 수반행렬(adjugate,
adjunct), 일차독립, 일차종속, Subspace, 기저, 차원, nullity, column
space and row space, 계수(rank), RankNullity 정리, 최소재곱해(least square
solution), Orthonormal basis, GramSchmidt 정규직교화 과정, QR 분해, 선형변환(Linear
Transformation), 표준행렬, 핵(kernel)과 치역(range), 동형사상(isomorphism), 고윳값, 고유벡터, 대각화(Diagonalization),
닮음 행렬(similar matrix), 직교대각화(orthogonally
diagonalizing), 고윳값분해(eigendecomposition), SVD(특이값 분해), 일반화된 역행렬(Pseudoinverse,
MoorePenrose Generalized Inverse), 이차형식(quadratic form), 주축정리(Principal axis
theorem), 함수, 극한(limit), 도함수(derivative)와 미분(differentiation),
뉴턴 방법(Newton’s Method with
gradient), 적분(Integral), 리만 합(Riemann sum), 외적(Cross Product), 벡터 함수, 편도함수(Partial
Derivative)와 그래디언트(gradient), Jacobian, 연쇄법칙(Chain Rule), 방향도함수(directional
derivative), Taylor 정리, 함수의 극대(Local Maximum)
and 극소(Local Minimum), Fermat’s theorem on
critical points, 임계점(critical point), 안정점(saddle point),
Gradient Descent Algorithm(경사기울기 하강법), 중적분(double integral,
multiple integral), Fubini의 정리의 개념을 이해하고 설명할 수 있으며 간단한 것은 손으로, 복잡한 것은 sage 또는 python 을 이용하여 계산하여 그 의미를 설명할 수 있다.
(3) 개인/동료와 같이 “본” 강좌를 학습하면서 배우거나 느낀 점은?
I am very glad that I had the opportunity
to once again get into this wonderful environment of the PBL class. I really
wanted to deepen my mathematical knowledge in the field of Artificial
Intelligence, and this summer semester, thanks to this subject (basic
mathematics for artificial intelligence), I have this opportunity. As I said
earlier, I really like the PBL technique, because thanks to it, each student
can actively participate in the educational process.
(4) What you contributed through this course (Q &
A and/or Inclass):
I tried to actively participate every
week in the QnA. Most of my QnA activity consists of answering questions,
coding, finalizing and summarizing lecture materials. In almost every post I
have tried to attach examples of solving problems by hand and using Sage or
Python(or both) code, as I believe that a visual stepbystep process of
solving problems in practice can greatly help me and other students in
understanding the topic.
(5) (5 pt) What is your most
important contribution or founding that you shared with others in QnA.(Quality)
To my mind, my most important contribution that I
shared with others was:
[Final OK by SGLee] [HW] 1주차 강의 내용 요약 및 정리 + Sage과 Python 실습(비교), 내용 요약 및 정리 by 박진형, 코딩실습 및 Finalized by 유가이 올렉산드르
Here Mr. 박진형 summarized all the material for the first week, and I
added Python and Sage code for deeper understanding the material. Also, my goal
was to compare the capabilities of Python and Sage in solving mathematical
problems. From this coding practice, we can conclude that it is much more
convenient and faster to apply Sage in a mathematical environment.
(6)
Number of Final OK by SG Lee Problems (and/or Completed Discussion/Question) in
Q&A that your name is included:
13
B. (Quantity 10pts) Quality of
Your Participation:
(1) Write what you especially remember while you are
doing A1, 2, 3.
I am very pleased that most of the
students in this class are very actively involved in the educational process.
Everyone is trying to share their knowledge in QnA, ask questions on an
incomprehensible topic, or help another student with a detailed answer. I also
try to be a part of this process and to participate as much as possible in the
QnA.
(2)
What did you learn or feel while learning Math for AI (Action Learning/PBL)
with your classmates?
I have greatly improved my mathematics
knowledge and also deepened my knowledge of programming. Thanks to the active
work of the Professor, TA and students, I significantly deepened my knowledge
of SVD, Newton's Method and Gradient Descent Method and learned a lot about the
way to apply them in practice.
Ch. 3: Personal
Reflection Note (20점)
Subject 
Basic Math for AI 
Major 
BioMechatronic Engineering 

Name 
Oleksandr Yuhai 
Year 
2018 2nd Semester 

Learning contents 
Part0(Power Method, MNIST, AlphaGo). Part1(벡터, 선형연립방정식, 행렬과 행렬식, 일차독립과 기저(basis) 및 차원(Dimension), 선형변환
(Linear Transformations), 고윳값, 고유벡터, 대각화(Diagonalization), SVD (특이값 분해, singular value decomposition), 이차형식(quadratic form)). Part2(일변수함수와 미적분, 다변수함수와 미적분) 

SelfChecking 

Activity 
Excellent 
Good 
Fair 

1. 
I have contributed
to generate ideas and facts needed to resolve the issue. 
● 



2. 
I proposed learning
issues associated with learning. 

● 


3. 
When I study alone,
I used a variety of learning materials. 
● 



4. 
I provide new
information and knowledge in this class. 

● 


5. 
I was actively
involved in the discussions. And I provided a lot of questions in order to
understand these discussions. 

● 


6. 
I have made a contribution
to the learning activities for our class. 
● 



※ Please record the following items by considering your learning
process. 1. Do you understand the most of contents of this
learning process? I understood at a
sufficient level most of the topics from Part 0 to Part 2. I also learned
many new methods of how to solve mathematical problems using programming
languages (Sage, Python). 2. What kind of learning materials have you used to
study? In the
process of studying Mathematics for Artificial Intelligence, I most often
used our professor's textbook and video lectures. But I also actively used
various books such as Linear Algebra and its applications by David C. Lay,
Calculus  Early Transcendentals by James Stewart, and others. I also used
useful information from various math forums, thematic videos on YouTube, math
blogs. I believe that all this was made possible through the use of the PBL
learning system. Usually,
I first get acquainted with the topic in our textbook book(Basic Math for AI
by 이상구 교수님 with 이재화
TA님), then I watch the lecture of the professor, after
that I look for more information on topics that are not entirely clear to me
in other lectures of the professor (Linear Algebra, Calculus, etc.) on the
site http: // matrix.skku.ac.kr/sglee/, after that for a deeper understanding
of the material, I look for information in other books (Linear Algebra and
its applications by David C. Lay, Calculus  Early Transcendentals by James
Stewart, etc.). After that, I begin to search for more extensive information
on the topic on the Internet (usually material related to practice). Finally,
I start to practice solving tasks from textbooks manually and using Sage or
Python (or both). After that, I usually post all the information I have
collected in the QnA section and then start trying to answer questions from
other students in QnA. 3. What did you learn through the learning
activities of this course? Through learning activities I realized the
importance of solving math problems not only by hand, but also using
programming languages such as Sage and Python. I also understood the
importance of active participation of students in the educational process. I
believe that thanks to the active participation of students and Professor in
the QnA, many even rather complex mathematical topics have become much
clearer. Even though I have already taken courses on Linear Mathematics and
Discrete Mathematics, I learned a lot of new things for myself from other
students (for example, the scope of SVD). Also, since I did not take courses
in Calculus, I learned a lot about such very interesting and important topics
as partial differentiation, and the Gradient Descent Algorithm. 4. What have you learned from the other colleagues? 5. SelfEvaluation for Q/A Activities My score: 97  Sometimes I
posted my posts to the ICampus a little late, as they took me a lot of time
to write, but I tried to post at least 4 posts(summarizing the lecture
materials, answering questions, finalizing, programming etc.) every week. 6. Evaluation for other students Almost
all students in our class actively participated in the development of our Basic
Math for AI class. I think that, thanks to the PBL system, many students were
able to become a support for each other in understanding a variety of
difficult mathematical topics. And I also think that thanks to the active
participation of students, the professor, and TA in the learning process,
many students were able to master most of the material from the first two
parts at a fairly good level (even students from the Humanities and Social
Sciences Campus who may not be familiar with Linear Algebra or Calculus
before). 









SelfEvaluation
Subject 
Basic
Math for AI 
Major 
BioMechatronic
Engineering 

Name 
Oleksandr Yuhai 



Evaluation Items 
Strongly
disagree 
Disagree 
Mostly disagree 
Mostly agree 
Agree 
Strongly agree 


1. I participated actively in both, online and offline classes. 





● 


2. I participated actively on a Q&A activity. 




● 



3. My question and replies made on Q&A are relevant. 





● 


4. Information provided by my activity was useful for other students in
the class. 




● 



5. I enthusiastically took into the consideration other students’
opinions or point of view. 





● 


6. I contributed to class by participating on Q&A discussions. 




● 



7. I am enthusiastic about taking other class with the same students I am
taking Basic Math for AI. 





● 

[Opinion] ► Satisfaction according to the SelfEvaluation I
think that I was quite active in QnA, summarizing up the lecture material,
practicing solving various problems manually and through the program code,
finalizing the solutions of other students and, if possible, answering
questions. Also, I think that, my work has helped other students to learn
something new in solving problems in Linear Algebra and Calculus. ► Sorrow according to the SelfEvaluation Sometimes
I posted my posts in the QnA section a little late (at the end of the week),
but I tried to make each of my posted posts as informative and useful as
possible. 











PeerEvaluation


Subject 
Basic Math for AI 

Colleague’s
name 
김범준 

Name
of evaluator 
Oleksandr
Yuhai 

Evaluation Items 
Strongly
disagree 
Disagree 
Mostly disagree 
Mostly agree 
Agree 
Strongly agree 


1. I participated actively in both, online and offline classes. 





● 


2. I participated actively on a Q&A activity. 





● 


3. My question and replies made on Q&A are relevant. 




● 



4. Information provided by my activity was useful for other students in
the class. 




● 



5. I enthusiastically took into the consideration other students’
opinions or point of view. 




● 



6. I contributed to class by participating on Q&A discussions. 





● 


7. I am enthusiastic about taking other class with the same students I am
taking Discrete Mathematics. 




● 


[Opinion] ► Satisfaction according to the SelfEvaluation A
student from the Faculty of Consumer Science, pretty quickly adapted to the
PBL system of this math related class and actively participated in QnA. I
also believe that during his studies he improved his mathematical and
programming (Sage) knowledge to a great extent. ► Sorrow according to the SelfEvaluation 없습니다. 










Flipped/PBL Action Learning Class
Evaluation
Details 
Evaluation 

Strongly
agree 
Agree 
Neutral 
Disagree 
Strongly disagree 

1. I
participated actively in Flipped/PBL classes. 

V 



2. Professor helped students to
demonstrate their abilities. 
V 




3. I
obtained new skills and knowledge due Flipped classes. 
V 




4. Through that process of learning I developed
my abilities of making hypothesis. 

V 



5. Through that process of learning I obtained or
developed leadership capabilities. 

V 



6. I learned multiple methods of solving
problems. 
V 




7. I am now able to use various materials needed
to study efficiently enough. 
V 




8. I developed my professionalism. 
V 




9. The process of learning I experienced is
similar to the real process of doing researches. 
V 




10. Evaluation of method of learning is relevant. 
V 

