Familiarity with Linear Algebra helps quite a bit while working on Machine Learning algorithms. As Software Programmers, we don’t get to do lot of serious Mathematics on regular basis tend to forget the principles. I would like to list some of the concepts here.
Matrices & Vectors
A Matrix is denoted by Number of Rows * Number of Columns. The above Matrix is normally called a 3 * 2 Matrix.
Similarly Vector is another often used Entity in Machine Learning algorithms. It’s special type of Matrix in the sense it always has only one column. It also called a n * 1 Matrix. A typical example of Vector would be:
Entries of Matrix or Vectors are denoted by where i is number of row and j is number of column. So in examples above, (Matrix) will be 4 and (Vector) will be 5.
We can perform subtraction in similar manner. One point to mention here is that we can perform these operations only on matrices of the same dimensions.
Scalar Multiplication & Division
We can also perform multiplication and divisions of Matrices and Vectors with real numbers as well. As you might have guessed, we take every number in the matrix and perform the corresponding operation with the provided number. Again an example will help:
Addition and subtraction can also be performed similarly.
Things get a bit interesting here. Firstly we can only perform multiplications between a Matrix and Vector when number of columns of Matrix equals number of rows of Vector. In mathematical terms m * n-dimensional Matrix can be multiplied with only n-dimensional Vector. The result of any Matrix – Vector multiplication is always a Vector. Let’s take help of an example to see how this is done.
As you can see, we are multiplying a 3 * 2 matrix with a 2 dimensional Vector and we get a 3 dimensional Vector in result.
A Matrix-Matrix multiplication can be considered as an extended case of Matrix-Vector multiplication: