This is the first of the two post series about Principal Component Analysis (PCA). This post lays down important knowledge bricks that are needed to understand the core principles of the PCA algorithm. The second post will discuss the actual implementation and its results by applying it to various data sets. Motivation The world is becoming more data driven than ever before. We collect large amounts of data from arbitrary sources.