8000 MNT Add pre-comit configuration by rth · Pull Request #16957 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]

Skip to content

MNT Add pre-comit configuration #16957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 23, 2020
Merged

Conversation

rth
Copy link
Member
@rth rth commented Apr 18, 2020

This adds a pre-commit setup to run a number of code style checks locally before each commit.

This is fully op-in, and only affect contributors who install pre-commit locally. Most of the below checks are already run in CI, so this allows to fail earlier. Among other things this would help not overwhelming CI during sprints.

It runs only on modified files and currently,

  • checks that yaml files have have valid syntax (e.g. CI config would fail with an error otherwise)
  • strip trailing whitespaces. Some editors already do this automatically. This can produce unrelated changes in edited files, but as long as everybody does it it should not result in merge conflicts. Trailing white-spaces are already marked in red in github diff, and currently sometimes reviewers ask to remove them manually. It's also a "W291 trailing whitespace" flake error.
  • fix the end of files (by making sure there is newline there, I think)
  • run flake8 (only for unused imports F401 for now, since we are generally not fully flake8 compatible) RFC On the relative harm of cosmetic changes #11336 (comment)
  • run mypy on changed files

When one does pre-commit install this install all of the above into a virtualenv, so in the long run it can simplify our contributing guide by not asking to install them manually. It also pins versions of these tools to avoid a situation where locally linters pass but fail in CI.

So far this doesn't add pre-comit in CI, and pre-commit remains fully optional for contributors. Personally I would would like to be able to use it though, and it makes sense to share the config.

For instance pandas has an extensive setup https://github.com/pandas-dev/pandas/blob/master/.pre-commit-config.yaml

cc @thomasjpfan @NicolasHug @adrinjalali

Copy link
Member
@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I see where this is going :p

Nits but LGTM. I tried locally and it works as expected.

rth and others added 4 commits April 18, 2020 16:00
Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
@rth
Copy link
Member Author
rth commented Apr 18, 2020

Thanks @NicolasHug I addressed your comments.

I think I see where this is going :p

Nowhere bad, just looking for reviewers who could approve this PR ^^

- repo: https://gitlab.com/pycqa/flake8
rev: 3.7.8
hooks:
- id: flake8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this follow the configuration in examples/.flake8 when linting examples?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well right now this only checks for one single error (F401: unused imports) which is also compatible with examples/.flake8. Now it's more a proof of concept for flake8, I'd be happy extend it once we have this pre-commit config file merged :)

Copy link
Member
@jnothman jnothman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This declarative approach to pre-commit hooks is neat.

Do these run on the whole code base, or on the diff files?

$ git add modified_files
$ git commit
$ pip install pre-commit
$ pre-commit install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might need to mention that occasionally (particularly when creating merge commits), you might need to use git commit -n to disable pre-commits.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What error message will users get the first time?
If it's explicit enough so that a simple google search give them the answer, I'd leave it out.

This section is aimed at new contributors and we need to be careful about not overloading with information (that was an issue in the previous version of the guide that we tried to trim down). Also I honestly doubt they'll ever remember "oh right I saw that in the UG"

@rth
Copy link
Member Author
rth commented Apr 19, 2020

Do these run on the whole code base, or on the diff files?

They only run on files that changed, but on the full file not just the diff.

What error message will users get the first time?
If it's explicit enough so that a simple google search give them the answer, I'd leave it out.

Well it's exactly the same error message they would get in CI: i.e. the error message from from flake8 or mypy saying that at Line N something is wrong. Ideally we should aim for the situation where the same linters are run in pre-commit and in CI. So it they disable pre-commit it would just later fail in CI and still need fixing.

For the currently added checks I don't think there is a use case where where one would need to disable pre-commit (worst case scenario is adding # type: ignore for mypy for lack of better solution). In general I think searching "pre-commit disable" or or running pre-commit --help (to find pre-commit uninstall) should yield enough information if needed.

@rth
Copy link
Member Author
rth commented Apr 19, 2020

(particularly when creating merge commits), you might need to use git commit -n to disable pre-commits.

Actually, you are right for merge commits. Ignore my earlier message. Added the sentence to the doc, it cannot hurt.

Copy link
Member
@adrinjalali adrinjalali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quite happy to have these 🍾

@thomasjpfan thomasjpfan changed the title Add pre-comit configuration MNT Add pre-comit configuration Apr 23, 2020
@thomasjpfan thomasjpfan merged commit e392bfd into scikit-learn:master Apr 23, 2020
gio8tisu pushed a commit to gio8tisu/scikit-learn that referenced this pull request May 15, 2020
Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
viclafargue pushed a commit to viclafargue/scikit-learn that referenced this pull request Jun 26, 2020
Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0