8000 sample_weight for lasso, elastic etc · Issue #3702 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

sample_weight for lasso, elastic etc #3702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
3 tasks done
szs8 opened this issue Sep 25, 2014 · 32 comments · Fixed by #22808
Closed
3 tasks done

sample_weight for lasso, elastic etc #3702

szs8 opened this issue Sep 25, 2014 · 32 comments · Fixed by #22808
Labels
Enhancement help wanted Moderate Anything that requires some knowledge of conventions and best practices module:linear_model

Comments

@szs8
Copy link
szs8 commented Sep 25, 2014 •
edited by lorentzenchr
Loading

I am using lasso for time-series data and I would like to decay old data with a weight vector. Any reason why this is not available for lasso?

Meanwhile, if I fit lasso with X = dot(sqrt(diag(weights)), X) and y = dot(sqrt(diag(weights)), y), would it mean the same thing?

Thanks!

Edit:
TODOs
@agramfort
Copy link
Member

you mean you want sample_weight support for Lasso and ElasticNet?

@szs8
Copy link
Author
szs8 commented Sep 25, 2014

Yes, exactly.

On Sep 25, 2014, at 11:24 AM, Alexandre Gramfort notifications@github.com wrote:

you mean you want sample_weight support for Lasso and ElasticNet?
—
Reply to this email directly or view it on GitHub.

@agramfort
Copy link
Member

it's rather trivial if you don't fit intercept or if you use dense data but
it's a pain with sparse + fit_intercept. Maybe we can avoid this usecase...?

@szs8
Copy link
Author
szs8 commented Sep 25, 2014

Yes, I can work around it but it is implemented for ridge now and I think it would be nice to have a consistent API.

@agramfort
Copy link
Member

feel free to send a PR

@amueller amueller added the Enhancement label Jan 22, 2015
@dwysocki
Copy link

I too use the Lasso for timeseries analysis, and have inputs with measurement uncertainties. Weights would be invaluable.

Just how difficult would it be to implement? I'd be willing to try my hand at it, if nobody else is (reading suggestions would be welcomed).

@agramfort
Copy link
Member
agramfort commented Oct 31, 2015 via email

@dwysocki
Copy link

You're referring to #5526?

I wonder if I can help somehow...

@giorgiop
Copy link
Contributor

You're referring to #5526?

Yes. Still working on it. This might actually not be a real bug, but I will be touching the base classes of linear models to simplify and clean the code in #5357.

@dwysocki
Copy link

I see. Godspeed!

@giorgiop
Copy link
Contributor

I think a weighted lass/lars would be welcome. I suggest not to wait until my work get merged, it may take a little while.

@dwysocki
Copy link

Oh? I got the impression the bug you're working on was blocking this.

Are there any pitfalls to watch out for then?

@mblondel mblondel changed the title Weights for lasso, elastic etc sample_weight for lasso, elastic etc Nov 2, 2015
@howthebodyworks howthebodyworks mentioned this issue Sep 6, 2016
Closed
@starfleetjames
Copy link
starfleetjames commented Dec 1, 2017 •
edited
Loading

As part of the "etc" in the question, also for validation_curve! This would probably just be a pass through argument to the actual model being run. In my case, sample_weight works fine for SVR but validation_curve doesn't accept it even when I'm running it on SVRs varying the gamma parameter.

@jnothman
Copy link
Member
jnothman commented Dec 3, 2017 via email

@starfleetjames
Copy link

@jnothman, I see a set_params method for SVR that would be perfect, but no similar method in validation_curve. Could be misunderstanding your suggestion though.

@jnothman
Copy link
Member
jnothman commented Dec 4, 2017 via email

@jnothman
Copy link
Member
jnothman commented Dec 4, 2017 via email

@mandeldm
Copy link
mandeldm commented Jan 8, 2018

Hi. Are there current plans to incorporate sample_weight into Lasso, ElasticNet etc. the way it is done in Ridge? Thanks.

@agramfort
Copy link
Member
agramfort commented Jan 11, 2018 via email

@iamDecode
Copy link

It does not seem like any PR is currently addressing this. I gave it a try but am not familiar enough with the codebase to implement it. Would be great to see this feature implemented tho!

@twiecki
Copy link
twiecki commented May 13, 2019

+1 that this would be useful feature.

@brian36
Copy link
brian36 commented Jul 8, 2019

+1 also need this

@thomasjpfan thomasjpfan added the Moderate Anything that requires some knowledge of conventions and best practices label Jul 13, 2019
@chnorn
Copy link
chnorn commented Jul 23, 2019

This would be super useful!

@jdthorpe
Copy link
jdthorpe commented Jul 31, 2019 •
edited
Loading

Correct me if I'm wrong (and I very well may be wrong), but my understanding is that for values X,y,and sample_weight this can already be achieved with

Gram = np.dot(X.T * sample_weight, X)
Xy = np.dot(X.T * sample_weight, y)
sklearn.linear_model.lars_path_gram(Xy, Gram, y.size, method="lasso")
# or equivalently
sklearn.linear_model.lars_path(X=None, y=y, Xy=Xy, Gram=Gram, method="lasso")

Wish I had time to write up a PR incorporating this into sklearn.linear_model.Lars and friends, but for someone who does, this is a hint at how to do it...

@rth rth mentioned this issue Sep 30, 2019
Merged
7 tasks
@lorentzenchr
Copy link
Member

Is anyone working on this? If not, I might give it a try.
As originally suggested by @szs8, I think a simple rescaling of X and y by sqrt(sample_weight) suffices (maybe up to a scaling factor like n_sample/np.sum(sw)), as is done for ridge:

scikit-learn/sklearn/linear_model/ridge.py

Line 437 in 5c9f090

X, y = _rescale_data(X, y, sample_weight)

Or am I missing something?

@lorentzenchr
Copy link
Member

The sparse case with fit_intercept=True seems to be more involved. I can't figure out a solution without modifying

scikit-learn/sklearn/linear_model/_cd_fast.pyx

Line 255 in 53082e9

def sparse_enet_coordinate_descent(floating [::1] w,

@lorentzenchr lorentzenchr mentioned this issue Nov 1, 2019
Merged
@lorentzenchr lorentzenchr mentioned this issue Nov 13, 2019
Closed
@nathanzhang3
Copy link

It will be very helpful to have sample_weight for ElasticNet and Lasso

@lorentzenchr lorentzenchr mentioned this issue Feb 16, 2020
Merged
@lorentzenchr
Copy link
Member

@szs8 @nathanzhang3 #15436 is merged and enables sample_weight for ElasticNet and Lasso with dense X as of release 0.23.

Should we close this issue or keep it open until ElasticNetCV and also sparse X have sample weight support?

@agramfort
Copy link
Member
agramfort commented Aug 8, 2020 via email
< A93C /div>

@biprateep
Copy link
Contributor

I see that sample_weight is available for Lasso, is there a way to make it available for LassoCV too?

@lorentzenchr
Copy link
Member

@biprateep #16449 addresses LassoCV and ElasticNetCV, but is not merged yet.

@MaxwellLZH MaxwellLZH mentioned this issue Jun 29, 2021
Closed
@lorentzenchr lorentzenchr mentioned this issue Nov 24, 2021
Closed
@lorentzenchr lorentzenchr mentioned this issue Mar 12, 2022
Merged
@lorentzenchr
Copy link
Member

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Enhancement help wanted Moderate Anything that requires some knowledge of conventions and best practices module:linear_model
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ENH add sample_weight to sparse coordinade descent lorentzenchr/scikit-learn
0