My Learnings in ML

t-SNE is a non-linear dimensionality reduction technique that is well suited as a visualization technique as its goal is to reduce the dimensionality to 2 or 3 dimensions. The ideal goal of visualization is to build a map in low dimensional space that reflects the similarities in the original space. Popular dimensionality reduction techniques like PCA are not very useful for this purpose because the goal of PCA is to maximize variance which is the same as trying to minimize the squared error between distances in the original data and the map and hence this leads to preserving large pairwise distance in the map. However large pairwise distances are not always reliable and on the other small distances between neighbors is reliable. Techniques relying upon preserving neighboring distance are Isomap, LLE, SNE and t-SNE. t-SNE is based upon the SNE (stochastic neighbor embedding) algorithm and places a Gaussian around every point in the high dimensional space and then measure the density

SVD is a popular approach for factorization of a matrix and has several applications in many domains. It proceeds by factorizing a given matrix A m x n as A = UΣV^T, where U is a m x m unitary matrix, i.e. U^TU=I, Σ is a m x n diagonal matrix consisting of the eigenvalues and V is a n x n unitary matrix. It is popularly used to find the rank k approximation of a given matrix. One of the main problems of using SVD for large matrices is the computation of the eigenvector decomposition. Lets see how the eigenvector decomposition can be carried out for large sparse matrices. Power Iteration : One of the simplest methods of computation of the largest eigenvalue of a large sparse matrix is the power iteration. It however only computes a single largest eigenvalue and may converge very slowly. The algorithm is as follows: Start with a random vector b0 b_{k+1} = Ab_k || Ab_k|| This algorithm converges to the largest eigenvalue under the assumption that there is a si