Low-rank matrix approximation with SVD – Part 2

In part-1 we have revising the internal workings of singular value decomposition.  Here in part 2 lets see the practical usage of SVD.

All source code and corpus used in this excise can be found on my github-Account.

Singular value decomposition (SVD) has a wide range of usage in the statistical study however it is more popular for its usage called Low-rank matrix approximation (dimension reduction).

Singular value decomposition

Low rank matrix approximation

The SVD can be used to approximate the high-dimensional matrix to low-dimensional matrix by a lowering the rank of the original matrix.

In other words the rank ‘k’ of a matrix, helps to produce a least-square matrix approximation from the first k singular vectors of U and V and the k corresponding singular values. This approximate optimal matrix is more similar to original matrix and produce the same result as that be could produced from processing original high-dimensional matrix. As the result are going same processing the approximated low-dimensional matrix will help in reducing computational and processing resources.

To visualize this Low-rank matrix approximation, let works with matrix that represents an image. Going further lets reduce the rank of a matrix and reproduce the image from a low rank matrix and check its quality. To do this excise I used R language to compute SVD and play with a matrix representing the image.

All source code and corpus used in this excise can be found on my github-Account.

 svd-low-rank-matrix-approximation

All source code and corpus used in this excise can be found on my github-Account.

With only 10% of the real data, we are able to create a very good approximation of the real data. Moreover, with this method we can remove noise and linear dependent elements by using only the most important singular values. This is very useful on data mining problems as it is hard to identify which elements in the data set will be useful or not for to statistical analysis.

Other notable usage of SVD is Pseudoinver, Solving homogeneous linear equations,latent semantic index and etc. In fact I have written a another blog on Latent Latent Semantic Analysis that uses SVD technique.


Post a Comment

Your email is never published nor shared. Required fields are marked *