No Bessel correction in PCA #7699
Comments
|
I just checked Murphy, Hastie/Tibshirani/Friedman, Bishop and Shalev-Shwartz/Ben-David (selected by being within arms reach) and none of them include the Bessel's correction in their PCA formulation. |
|
This is interesting. Thinking about it, I could see where a Bessel correction might be applied or not applied based on the particular application. The Bessel correction serves to decrease the bias in inferred parameters at the expense of increasing the mean squared error. In a statistical modeling context (where you're attempting to, say, find an unbiased estimator of some latent parameter) this makes sense. In a machine learning context (where you're attempting to create a model with the best predictive power, where "best" often involves a mean-squared-error metric) a Bessel correction is probably not warranted. It goes back to the distinction raised in Breiman's Statistical Modeling: the Two Cultures. FWIW, in our astro textbook we define PCA using the Bessel correction for the covariance matrix. I'd never thought about it deeply before, but I imagine this is more of a standard in the astro community due to the fact that we generally use PCA in a statistical modeling sense (i.e. looking for meaning in the eigenvectors). |
|
@jakevdp do you think it warrants a parameter in PCA? We don't even have one in |
Perhaps, but in the vast majority of cases it's not going to make much difference at all, so I'd be tempted to leave it as-is, perhaps with a note in the docstring. |
|
yeah ok lets add a note to the docstring. |
|
Hello, |
Description
No big deal just missing Bessel's correction
Steps/Code to Reproduce
Expected Results
Actual Results
Versions
The text was updated successfully, but these errors were encountered: