Linear Algebra Involved
Given, the data set X(mxn matrix) where m is the number of measurement types and n is the number of samples.Mathematically ,the goal is to find some orthonormal matrix P in Y=PX such that is a diagonal matrix.The rows of P are the principal components of X.
Let’s state some elementary linear algebra theorems which will be helpful in formulating the above formula in a more meaningful way.
- The inverse of an orthogonal matrix is its transpose.
- For any matrix , , are symmetric.
- A matrix is symmetric if and only if it is orthogonally diagonalizable.
- A symmetric matrix is diagonalized by a matrix of its orthonormal eigenvectors i.e ( where D is the diagonal matrix and E is the matrix with eigenvectors of A arranged as columns)
Now,we select matrix P s.t each row is an eigenvector of
Therefore,the diagonal value of is the variance of X along
- In practice,computing PCA of a data set X entails:subtracting off the mean of each measurement type .(PCA doesn’t necessarily give a unique answer.For example if the data matrix is of Temperature,then it might give a different P for Celsius and Fahrenheit measurements.To minimize this error ,mean is subtracted from each measurement.In time series analysis we normally z square the data to achieve the same goal i.e subtrat the mean and divide by std. deviation)
- and computing the eigenvectors of