8000 Unclear warning message for ElasticNet(l1_ratio=0) · Issue #8233 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Unclear warning message for ElasticNet(l1_ratio=0) #8233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
NelleV opened this issue Jan 25, 2017 · 8 comments
Closed

Unclear warning message for ElasticNet(l1_ratio=0) #8233

NelleV opened this issue Jan 25, 2017 · 8 comments
Labels
Easy Well-defined and straightforward way to resolve Sprint

Comments

@NelleV
Copy link
Member
NelleV commented Jan 25, 2017

I am currently attempting to fit a Non Negative Least Square with an l2 penalization without having to reimplement it.
The solution I have adopted is to use ElasticNet(positive=True, l1_ratio=0), but this yields confusing warning messages.

Description

I am using ElasticNet with the parameter l1_ratio=0 and positive=True. It yields a confusing warning message, as the warning message refers to a parameter called alpha, supposedly set to 0, but the only alpha parameter the user has access to is not set to 0

In [8]: %run test_elasticnet_as_ridge.py
/home/nelle/.envs/goldwrap/lib/python3.5/site-packages/sklearn/linear_model/coordinate_descent.py:470: UserWarning: Coordinate descent with alpha=0 may lead to unexpected results and is discouraged.
  positive)

In [9]: est.alpha
Out[9]: 1.0

My assumption is that ElasticNet is not meant to be used with l1_ratio=0, which should be explictily mentioned, though this is as well unclear as the documentation mentions the following: Currently, l1_ratio <= 0.01 is not reliable, unless you supply your own sequence of alpha. and alpha is a float, set by default to 1.

Steps/Code to Reproduce

from sklearn import linear_model
import numpy as np

n = 1000
X = np.random.randn(n, 3)
beta = np.array([2, 3, 2])

Y = np.dot(X, beta) + np.random.randn(n)
est = linear_model.ElasticNet(positive=True, l1_ratio=0)
est.fit(X, Y)
@NelleV
Copy link
Member Author
NelleV commented Jan 25, 2017

@agramfort git blames you, at least for the unclear documentation par :D

@agramfort
Copy link
Member

warning comes from

    if alpha == 0:
        warnings.warn("Coordinate descent with alpha=0 may lead to unexpected"
            " results and is discouraged.")

in cd_fast.pyx where alpha is the L1 regularization so self.alpha * self.l1_ratio which is zero in your code.

the warning should say something like

    if alpha == 0:
        warnings.warn("Coordinate descent with no L1 regularization may lead to unexpected"
            " results and is discouraged.")

or

    if alpha == 0 and beta == 0:
        warnings.warn("Coordinate descent with no regularization may lead to unexpected"
            " results and is discouraged.")

however thinking about it will depend on the conditioning of the prob 8000 lem....

the original motivation for this warning was the p >> n setup with no reg which will converge very slowly.

@vene
Copy link
Member
vene commented Jan 28, 2017

But is there actually any convergence problem if alpha == 0 but beta > 0? I would imagine some L2 regularization should make things OK (i.e. in line with @agramfort's second proposal)

@agramfort
Copy link
Member
agramfort commented Jan 29, 2017 via email

@NelleV
Copy link
Member Author
NelleV commented Feb 7, 2017

I can work on this.
I don't know whether there is a problem, but the optimization algorithm is not very well adapted for a constrained ridge regression. I ended up reimplementing a simple and inefficient of solving the same optimization problem using l-bfgs-b. I think an active set algorithm would be faster.

i'll still fix this slight issue.

@agramfort
Copy link
Member
agramfort commented Feb 9, 2017 via email

@amueller amueller added Easy Well-defined and straightforward way to resolve Need Contributor Sprint labels Mar 4, 2017
@Sentient07
Copy link
Contributor

No news for a while, can I take this if there is no on going work?

@NelleV
Copy link
Member Author
NelleV commented Mar 7, 2017

You can take this ticket if you'd like.

@Sentient07 Sentient07 mentioned this issue Mar 9, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Easy Well-defined and straightforward way to resolve Sprint
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@agramfort @NelleV @vene @amueller @Sentient07
0