10000 Weighted ridge regression regularization variable is dependent on sample weight magnitude · Issue #27285 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Weighted ridge regression regularization variable is dependent on sample weight magnitude #27285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
elliot-kulakow-ent opened this issue Sep 4, 2023 · 2 comments · Fixed by #28119
Closed

Weighted ridge regression regularization variable is dependent on sample weight magnitude #27285

elliot-kulakow-ent opened this issue Sep 4, 2023 · 2 comments · Fixed by #28119
Labels
Documentation

Comments

@elliot-kulakow-ent
Copy link

Describe the issue linked to the documentation

When doing weighted ridge regression, the value of the regularization parameter for a particular solution is dependent on the sample weight vector due to scaling in the implementation. Scaling should either be according to the weighted average instead of just a simple multiplication, or the documentation should specify that the weight vector should sum to 1. This may affect other learners as well. As is, this is unstable in a cross validation context as the data magnitude may change between folds.

Suggest a potential alternative/fix

Specify the weight vector should sum to one, or fix the underlying behavior to avoid the dependence.
@glevv
Copy link
Contributor
glevv commented Sep 15, 2023

It is somehow connected to #26848

Maybe there should be a discussion about some general accepted way of dealing with sample weights in different algorithms ( scaling them to sum up to 1, etc)

@glemaitre
Copy link
Member
glemaitre commented Sep 16, 2023 •
edited
Loading

I recall some discussion here regarding the scaling (or not) of sample-weight: #15657

@glemaitre glemaitre removed the Needs Triage Issue requires triage label Sep 16, 2023
This was referenced Jan 12, 2024
Closed
Closed
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

DOC Provide info regarding impact of sample weights scale sperret6/scikit-learn
3 participants
@glemaitre @glevv @elliot-kulakow-ent
0