8000 Improve speed plot_permutation_tests_for_classification.py · Pull Request #21649 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Improve speed plot_permutation_tests_for_classification.py #21649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from Nov 16, 2021
Merged

Improve speed plot_permutation_tests_for_classification.py #21649

merged 2 commits into from Nov 16, 2021

Conversation

ghost
Copy link
@ghost ghost commented Nov 12, 2021

#21598 @adrinjalali

Changed the number of features and permutations
permut_before
permut_after

@TomDLT
Copy link
Member
TomDLT commented Nov 12, 2021

Thanks for the PR.
I would just change the number of uncorrelated features to e.g. 20, because there is little need to use so many in this example.
However, I would keep the number of permutations to 1000 to have a reasonable p-value estimate.

@ghost
Copy link
Author
ghost commented Nov 13, 2021

@TomDLT Allright, agreed

@ghost
Copy link
Author
ghost commented Nov 13, 2021

Even better time improvement from 1:44 min (before) to 0:11 min (after)

@adrinjalali adrinjalali changed the title Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed plot_permutation_tests_for_classification.py Nov 13, 2021
@adrinjalali adrinjalali mentioned this pull request Nov 13, 2021
Closed
41 tasks
@adrinjalali
Copy link
8000 Member

Could you also check if using another model other than SVC could speed up the process?

@adrinjalali
Copy link
Member

@thomasjpfan re number of unrelated features, you happy with this?

@ghost
Copy link
Author
ghost commented Nov 13, 2021

@adrinjalali Yes, I also checked One-vs-One Logistic Regression, One-vs-All Logistic Regression and Multinomial Logistic Regression...all three were significantly slower than SVC.

@adrinjalali
Copy link
Member

I'd also check HistGradientBoosting*

@ghost
Copy link
Author
8000
ghost commented Nov 15, 2021

@adrinjalali Just checked it, also way worse (12 sec for LogReg vs 10 min HGB)...so far Logistic Regression seems unreached by other models

@TomDLT TomDLT changed the title Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed plot_permutation_tests_for_classification.py Improve speed plot_permutation_tests_for_classification.py Nov 16, 2021
@TomDLT TomDLT merged commit 263181a into scikit-learn:main Nov 16, 2021
@ghost ghost deleted the speed_increased_example_permutation branch November 16, 2021 17:32
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Nov 22, 2021
@glemaitre
ENH Improve speed plot_permutation_tests_for_classification.py (sciki…
aba14c8 
…t-learn#21649)
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Nov 29, 2021
@glemaitre
ENH Improve speed plot_permutation_tests_for_classification.py (sciki…
2c4fc7c 
…t-learn#21649)
samronsin pushed a commit to samronsin/scikit-learn that referenced this pull request Nov 30, 2021
@samronsin
ENH Improve speed plot_permutation_tests_for_classification.py (sciki…
4a1f772 
…t-learn#21649)
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Dec 24, 2021
@glemaitre
ENH Improve speed plot_permutation_tests_for_classification.py (sciki…
09cd79d
7398 
…t-learn#21649)
glemaitre pushed a commit that referenced this pull request Dec 25, 2021
@glemaitre
ENH Improve speed plot_permutation_tests_for_classification.py (#21649)
3cebf71
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
No Changelog Needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@TomDLT @adrinjalali
0