Logistic regression regularization penalization is applied to intercept #10626
Comments
|
this is a documented aspect of the liblinear solver, and we are in the
process of mitigating it by changing the default solver. You may submit a
PR improving the documentation in the meantime.
In short: it sucks, we know.
|
|
5 years have passed. Default solver is still liblinear, and no hint about penalyzing intercept in documentation. |
|
@avidale we always appreciate PR to help us improving the documentation. If you have some time it would be much appreciated. |
|
we intend to deprecate the default in the coming release. for a long time,
liblinear was the only option we had implemented. you're right, we should
have deprecated the default sooner. If logistic regression was the only
thing we were maintaining here, I expect we would have. But the job of
maintaining this package is much bigger than that.
|
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Description
The logistic regression penalization seem to be applied to both weights and intercept. While it is not really a bug per say, I think this behavior is really counter intuitive (and there is this post on stack overflow suggesting that this opinion is shared https://stats.stackexchange.com/questions/302986/how-avoid-regularizing-intercept-in-scikits-logisticregression)
It would be good to either correct it or give it as an option to pass through the class.
If there is no plan to correct it, I suggest to maybe change a little bit the documentation. The equation shows that only the weights are regularized which is misleading:
http://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
The text was updated successfully, but these errors were encountered: