Data and Matrices

[K-MOOC] Introductory Mathematics for Artificial Intelligence

$.\cover.png$

Translated by

Sang-Gu LEE with Youngju NIELSEN, Yoonmee HAM

from the original Korean text written by

Sang-Gu LEE with Jae Hwa LEE, Yoonmee HAM, Kyung-Eun PARK

Part Ⅱ. AI and Matrix

2 Data and Matrices

Linear algebra is a branch of mathematics concerning matrices, which is known to be the most useful tool. Solving most applied mathematics problems

relies on matrix computations. In Part 2 <AI and Matrices>, we will learn the fundamentals of linear algebra required for artificial intelligence,

including singular value decomposition (SVD), which is essential for dimension reduction of data.

2.1 Ordered tuples and vectors

2.2 Vector operations

2.3 Matrices and tensors

2.4 Matrix operations

2.5 Rules for matrix operations

2.1 Ordered tuples and vectors

Data can be represented as an ordered pair (or n-tuple). For example, the information on height, weight, age, sex of a person

can be considered as ‘Data for a person’. This data can be expressed as an ordered 4-tuple. Each number is called a component of the data.

Name	Height (㎝)	Weight (㎏)	Age	Gender (1: Male, 2: Female)		Data Representation
Kim	160	80	19	1	→	(160, 80, 19, 1)
Lee	170	70	27	2	→	(170, 70, 27, 2)
Park	180	56	30	1	→	(180, 56, 30, 1)

Two-dimensional (or three-dimensional) data consisting of two (or three) components represents a point on the coordinate plane (or coordinate space ).

It is not easy to visualize four or higher dimensional data, but we can think of it as a point lying in a higher-dimensional space .

For example, a data whose components are and represents a point on the coordinate plane.

An arrow whose starting point is the origin and the endpoint is is called a vector,

whose components are consist of (real) numbers, which is called a scalar.

그림입니다.
원본 그림의 이름: CLP00004e6c0002.bmp
원본 그림의 크기: 가로 232pixel, 세로 240pixel 그림입니다.
원본 그림의 이름: CLP00001bd00fe5.bmp
원본 그림의 크기: 가로 303pixel, 세로 284pixel

2.2 Vector operations

The following operations can be defined for vectors (on ).

(1) Scalar multiplication: For a scalar and a vector in ,

, [ ]

(2) Vector addition: For two vectors and in ,

, []

● Properties of Vector Operations

For any vectors , , in and scalars and , the followings hold.

①

②

③ [Zero vector means a vector whose components are all zeros.]

④ [Negative vector of ) means ]

⑤

⑥

⑦

⑧

Compute and when , and .

Solution.

Randomly generate vectors and scalars and then check the validity of Vector additions and Scalar multiplications

of those vectors and scalars in http://matrix.skku.ac.kr/KOFAC/.

2.3 Matrices and Tensors

The data of the height, weight, age, and sex of many persons can be gathered and arranged in a rectangular array called a Matrix.

This matrix's first row is the data for the first person and the second row for the second person, etc. The first row tells us

that he is a person with a height 160, weight 80, age 19, and a male. So, we can create a 3 by 4 matrix whose rows are the data

of each person. In this matrix, rows are "horizontal" collections of numbers and columns are "vertical" collections.

A row vector can be thought of as a matrix and a column vector as an matrix. So, we have constructed a matrix from the given data.

A matrix can be very useful for representing a digital image. For example, a small grid that appears when a digital image is enlarged is called a pixel,

and each pixel contains a number indicating the brightness of the image. We can think of brightness as numbers. Therefore, a grayscale image

can be represented as a matrix, and a color image can be thought of as a three-dimensional matrix represented by three channels,

<Red, Green, and Blue (RGB color)>. We can also think of a color image as a cube-shaped matrix. This is called a ‘3-dim Tensor’.

In other words, a grayscale image can be expressed as a matrix by matching components of the matrix to the brightness of the image

in such photographic images. And a color image can be thought of as a three-dimensional matrix consists of overlapping three components,

<Red, Green, and Blue>. Information on grayscale images and color images can be found.

묶음 개체입니다.

[Grayscale image] https://nrsyed.com/2018/02/17/kernels-in-image-processing/

[Color image] https://lisaong.github.io/mldds-courseware/01_GettingStarted/numpy-tensor-slicing.slides.html

☞ Note <What is a Tensor?>

Machine learning systems are using tensors as their basic data structure. The well known Google's TensorFlow takes its name from this term Tensor.

So what is a Tensor? In short, it can be said to be a container, which is a storage that can put data together, and since most of them deal with numeric data,

it can be understood as a container for numbers. We can think of a 1-dim Tensor as a Vector, a 2-dim tensor as a Matrix,

We can think of a 1-dim Tensor as a Vector, a 2-dim tensor as a Matrix, and a matrix's generalized form as a tensor.

2.4 Matrix Operations

Three matrix operations are defined as follows.

(1) Scalar multiplication

(2) Vector addition (Two matrices must have the same size.)

(3) Matrix product (matrix multiplication)

(For a matrix multiplication of two matrices, the number of columns in the first matrix must be equal to the number of rows in the second matrix.)

Ex .

2.5 Rules for Matrix Operations

[Law of Matrix Operations] Suppose that the matrices are appropriately sized matrices and are scalars.

①

②

③

④

⑤

⑥

⑦

⑧

⑨

⑩

Calculate , , and for the given , and .

, ,

Solution.

Randomly generate matrices and scalars to check matrix addition, real multiplication, and matrix product.

◩ Open Problem 5

Perform matrix operation by applying the operations on vectors and matrices learned above to matrices you find in other textbooks.
[ Refer: http://matrix.skku.ac.kr/LA/Ch-3/ ]

Here is an example: let's use a matrix to explain the relationship between sleeping hours, exercise hours, calorie intake, and the weight

and blood pressure of a person.

We can define a matrix expressing the effect of hours of sleeping, hours of exercise, and calorie intake on his weight

and blood pressure as follows.

(An -dimensional vector is an matrix which is a row vector, and an matrix is a column vector. In other words, a vector can be understood

as a special case of a matrix. We used this structure because a patient's hours of sleeping, hours of exercise, and calorie intake

may affect his/her weight and blood pressure.

그림입니다.

[Source: Prof. Ho-Sung Nam at Korea University]

그림입니다.
원본 그림의 이름: CLP00004ef00001.bmp
원본 그림의 크기: 가로 262pixel, 세로 216pixel =

Let’s assume that the following matrix is created by investigating the change in weight and blood pressure according to Kim's

and many other’s average daily hours of sleeping, hours of exercise, and calorie intake (with other related health information).

If this matrix is multiplied by the health information of a patient whose data tells 8 hours of sleeping, 1 hour of exercise, and 1,500 calorie

intake in a day, the expected weight of this patient is 71.9 kg and the expected blood pressure of this patient is 115.83 mmHg(Torr).

We could get the matrix based on patients’ health information and their weights and blood pressures. We can now use the matrix to figure out

what will be a patient’s expected weight and blood pressure only using the information about sleeping hours, exercise hours, and calorie intake.

[Understanding deep learning using matrix]

http://www.polymath.co.kr/contents/view/18142

☞ Note If the observed value of body weight and blood pressure has a significant difference from the predicted value, then this matrix must be modified.

We can find the modified matrix by slightly adjusting the components of the matrix to match the predicted value with the observed values.

It can be done by <Backpropagation> algorithm in deep learning.

(4) Transpose of a matrix: The transpose of a matrix is simply a flipped version of the original matrix. We can transpose a matrix

by switching its rows with its columns. For example, if is a matrix, then is a matrix.

For matrices and scalar , the following properties of a transpose matrix hold.

①

②

③ http://matrix.skku.ac.kr/sglee/linear/ocu/img/pf1-3-4.gif

④

Find the transpose matrix for each of the following matrices.

Solution.

(5) Diagonal matrix: A diagonal matrix is a matrix in which the entries outside the main diagonal are all zero. The diagonal matrix

with the main diagonal has values of is written as below.

diag

(6) Identity matrix: The identity matrix is a square matrix that has 1's along the main diagonal and 0's for all other entries.

Multiplying any matrix by the identity results in the matrix itself.

For an matrix ,

(7) Triangular matrix: A triangular matrix is a special kind of square matrix. A square matrix is called lower triangular if all the entries

above the main diagonal are zero. Similarly, a square matrix is called upper triangular if all the entries below the main diagonal are zero.

A triangular matrix looks like below.

묶음 개체입니다.

an upper triangular matrix a lower triangular matrix

(8) Symmetric matrix: A symmetric matrix is a square matrix that is equal to its transpose.

(9) Inverse matrix of : An matrix is called invertible (also nonsingular) if there exists an matrix such that .

If this is the case, then the matrix is uniquely determined by and is called the inverse of , denoted by .

A square matrix that is not invertible is called singular (noninvertible). In this case, there is no , such that .

For invertible matrices of and nonzero scalar , the followings hold.

① If is invertible, then is also invertible and .

② is invertible and .

http://matrix.skku.ac.kr/sglee/linear/ocu/img/pf1-5-2.gif

③ is invertible and

④ [Proof: ]

Find the inverse matrix of .

Solution. We can find the inverse matrix of when

Therefore, the inverse matrix of is

Find out whether is invertible.

Therefore, is not an invertible matrix. ■

◩ Open Problem 6

Find bigger than matrix from the internet or other textbook and check whether transpose and inverse matrices exist.

If those exist, find out what those are.

Homework Week 2

Add your comment in QnA on what you have learned from the activity of your summary/practice/questions/ answers.

http://matrix.skku.ac.kr/math4ai/PBL-Record/

http://matrix.skku.ac.kr/2020-Math4AI-PBL/

[Reference] More informations on Inverse matrix in

http://matrix.skku.ac.kr/K-MOOC-LA/cla-week-3.html