8000 Pin dependencies version in CI with lock files with a bot that auto-updates the lockfiles · Issue #22425 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]

Skip to content
Pin dependencies version in CI with lock files with a bot that auto-updates the lockfiles #22425
Closed
@lesteve

Description

@lesteve

Motivation

  • avoid CI breakage due to dependencies updates (recent examples: setuptools update, pytest7 update, coverage 6.3 update, very likely others ...)
  • avoid painful debugging when a breakage happens to figure out what is the cause of the problem and generally end up after some time that a dependency has changed
  • avoid breaking everyone PRs which is confusing for contributers

Ideas for implementation

  • have environment.yml files for each build rather than versions defined in azure/Circle .yml (for the build using build, we can either use poetry or conda-lock is supposed to support pip section in the environment.yml although I have not managed to get it to work yet)
  • have a script using conda-lock to regenerates conda lock files for each build based on their environment.yml
  • have a bot that runs this script updates the lock files and creates a PR, it seems like using an approach using GH action like Hypothesis does could work:
    HypothesisWorks/hypothesis@c96bbc8.
  • Have this PR merged if CI passes
  • only one PR opened at a time to avoid generating many PRs with the same diff. If there is already an open PR (based on a github label for example), push -f to the PR branch

Where to start

I'll probably try out this idea on one build first (e.g. one using conda-forge) to test it out, see how that works, and go from there.

cc @thomasjpfan in case you have some thoughs about this or you have already started working on something similar

Rejected alternatives

In an ideal workd we would "just" use Dependabot but it can spam forks with PRs. Numpy has given up on it see for example this

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0