Principal Component Analysis
Many approaches exist for reducing the dimension of feature vectors while still optimizing model evaluations. The subset selection approach is very useful and regularly applied. On the other hand, this approach may not reveal underlying relationships between the features or describe why certain features work well together while others do not. To do this, it is necessary to develop algorithms and compute recipes for mixing the most relevant features. Principal Component Analysis (PCA) is arguably one of the popular methodologies for achieving this goal.
Geometric Interpretation
Principal components analysis (PCA) projects the data along the directions where the data varies the most.
The first direction is decided by 
corresponding to the largest eigenvalue 
.
The second direction is decided by 
corresponding to the second largest eigenvalue 
.
The variance of the data along the principal component directions is associated with the magnitude of the eigenvalues.
Choice of How Many Components to Extract
Scree Plot – This is a useful visual aid that shows the amount of variance explained by each consecutive eigenvalue.
The choice of how many components to extract is fairly arbitrary.
When conducting principal components analysis prior to further analyses, it is risky to choose too small a number of components, which may fail to explain enough of the variability in the data.