feat: migrate to uv for faster dependency management #3388
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Generate uv.lock lockfile for deterministic builds (179 packages) - Update Python requirement to >=3.9 (EOL 3.8, numpy 1.26+ compatibility) - Add numpy>=1.26.0 constraint for Python 3.12+ support - Add helpful comment showing both pip and uv usage Key insight: uv fully supports [project.optional-dependencies]! - No need for [dependency-groups] duplication - Works with both: pip install -e '.[dev]' and uv sync --extra dev - Single source of truth, no maintenance burden Benefits: - 10-100x faster dependency resolution with uv - Deterministic builds via uv.lock - Full backward compatibility - pip still works perfectly - Aligns with tested Python versions (3.9, 3.14 in CI) - No dependency drift risk
- Copy uv binary from official ghcr.io/astral-sh/uv image - Copy pyproject.toml and uv.lock first for better layer caching - Replace pip with uv pip install for dependency installation - Split dependency installation and package installation into separate layers - Use --system flag to install into system Python (no venv needed in container) Benefits: - 10-100x faster dependency resolution during builds - Better Docker layer caching (dependencies cached separately from source) - Deterministic builds using uv.lock - Smaller image rebuild time when only source code changes
Updated workflows to use uv for faster CI/CD: - unit_tests.yml: Use uv sync and uv run for tests - lint_code.yml: Add uv setup, use uv pip install for dependencies - mypy.yml: Use uv sync and uv run for type checking - integration_test.yml: Update Python 3.8→3.9, use uv pip install Changes: - Add astral-sh/setup-uv@v5 action after setup-python - Replace 'pip install -e .[dev]' with 'uv sync --all-extras' - Replace 'pip install' with 'uv pip install --system' - Replace direct command execution with 'uv run <command>' - Remove pip cache since uv manages its own cache - Update Python version from 3.8 to 3.9 (min supported version) Note: Using --all-extras instead of --group because we're using [project.optional-dependencies] for compatibility with both pip and uv. Benefits: - 10-100x faster dependency resolution in CI - Automatic caching of uv dependencies by setup-uv action - Consistent environment between local dev and CI - Reduced CI workflow execution time
Updated documentation to recommend uv as the primary installation method: README.md changes: - Add new 'Using uv (recommended)' section first - Include uv installation command - Reorganize installation sections (uv, pip, conda) - Add uv examples for development version installation - Maintain backward compatibility with all existing methods CLAUDE.md changes: - Reorganize commands by workflow (install, test, lint, run) - Add 'Using uv' subsections for all command categories - Show both uv and pip/traditional methods side-by-side - Mark uv as recommended with performance note (10-100x faster) - Use 'uv sync --all-extras' (works with [project.optional-dependencies]) Benefits: - Users can easily adopt uv for faster dependency management - Clear migration path from pip to uv - Full backward compatibility maintained - Single dependency list in pyproject.toml (no duplication)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Migrates MultiQC to use uv for 10-100x faster dependency management while maintaining full backward compatibility with pip.
Design Decision: Using
[project.optional-dependencies]OnlyThis migration uses
[project.optional-dependencies]instead of the newer[dependency-groups](PEP 735) because:[project.optional-dependencies]via--extraor--all-extrasflagsThe only "cost" is typing
--all-extrasinstead of--group dev, which is negligible compared to avoiding maintenance complexity.Changes
Commit 1: Core uv setup
Commit 2: Update Dockerfile
Commit 3: Update CI workflows
Commit 4: Update documentation
Commit 5: Update .gitignore
Benefits
Testing
uv sync --all-extrasworks (installed 172 packages)uv run pytest --versionverifieduv run pre-commit --versionverifiedUsage
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com