8000 test_ridge_regression_unpenalized is unstable · Issue #23177 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

test_ridge_regression_unpenalized is unstable #23177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jeremiedbb opened this issue Apr 21, 2022 · 6 comments
Closed

test_ridge_regression_unpenalized is unstable #23177

jeremiedbb opened this issue Apr 21, 2022 · 6 comments
Labels
Blocker module:linear_model module:test-suite everything related to our tests
Milestone
1.1

Comments

@jeremiedbb
Copy link
Member

see https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=41172&view=logs&jobId=c0f56aa6-924f-525c-3d4d-cdeecf409e66&j=c0f56aa6-924f-525c-3d4d-cdeecf409e66&t=f81f9209-4a58-571a-6d88-f139846a65e1

cc/ @lorentzenchr
@jeremiedbb jeremiedbb added this to the 1.1 milestone Apr 21, 2022
@lorentzenchr
Copy link
Member

On one side, I feel guilty for all the work I've created by introducing those tests. On the other side, the shortcomings of the solvers become evident and - to formulate positively - it seems that sag and saga have room for improvement.

@jeremiedbb
Copy link
Member Author
jeremiedbb commented Apr 21, 2022 •
edited
Loading

What I don't get is that these failures are not deterministic. It's like there's some kind of randomness (but very subtle) with sag and saga. Although for the failure reported here, it happened in 2 CI jobs and the computed coefs are exactly the same in the 2 jobs

@lorentzenchr
Copy link
Member

Should we open an issue for this sag/saga?
@jeremiedbb Are you working on this issue or shall I open a PR today evening or tomorrow?

@jeremiedbb
Copy link
Member Author

@lorentzenchr I'm not sure what to do so if you have an idea about how to make it more robust go ahead.

Should we open an issue for this sag/saga?

Probably. I mean it would be interesting to better understand what we can expect with these 2 solvers

@lorentzenchr lorentzenchr mentioned this issue Apr 21, 2022
Open
@jeremiedbb
Copy link
Member Author

I probably found the source of the randomness (see #23181). At least now the CI runs should be reproducible :)

@jeremiedbb
Copy link
Member Author

With the randomness of the CI, this failure should not appear anymore. Since the global_random_seed fixture has been designed to consider a test passed if it passes for the 0-99 seeds, accepting that failures could happen for some other seed, I think we can consider this a non-issue. Let's close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Blocker module:linear_model module:test-suite everything related to our tests
Projects
None yet
Milestone
1.1
Development

No branches or pull requests
2 participants
@lorentzenchr @jeremiedbb
0