in

# Probabilistic View of Principal Part Evaluation | by Saptashwa Bhattacharyya | Jul, 2023

One of many primarily used dimension discount strategies in information science and machine studying is Principal Part Evaluation (PCA). Beforehand, We now have already mentioned a number of examples of making use of PCA in a pipeline with Support Vector Machine and right here we are going to see a probabilistic perspective of PCA to supply a extra strong and complete understanding of the underlying information construction. One of many greatest benefits of Probabilistic PCA (PPCA) is that it may deal with lacking values in a dataset, which isn’t doable with classical PCA. Since we are going to focus on Latent Variable Mannequin and Expectation-Maximization algorithm, it’s also possible to test this detailed post.

What you’ll be able to count on to be taught from this put up?

1. Quick Intro to PCA.
2. Mathematical constructing blocks for PPCA.
3. Expectation Maximization (EM) algorithm or Variational Inference? What to make use of for parameter estimation?
4. Implementing PPCA with TensorFlow Chance for a toy dataset.

Let’s dive into this!

## 1. Singular Worth Decomposition (SVD) and PCA:

One of many main vital ideas in Linear Algebra is SVD and it’s a factorization method for actual or complicated matrices the place for instance a matrix (say A) might be factorized as:

the place U,Vᵀ are orthogonal matrices (transpose equals the inverse) and Σ could be a diagonal matrix. A needn’t be a sq. matrix, say it’s a N×D matrix so we are able to already consider this as our information matrix with N situations and D options. U,V are sq. matrices (N×N) and (D×D) respectively, and Σ will then be an N×D matrix the place the D×D subset will likely be diagonal and the remaining entries will likely be zero.

We additionally know Eigenvalue decomposition. Given a sq. matrix (B) which is diagonalizable might be factorized as:

the place Q is the sq. N×N matrix whose ith column is the eigenvector q_i of B, and Λ is the diagonal matrix whose diagonal parts are the corresponding eigenvalues.