Migrate to uv package manager #13622
Conversation
Consolidate Python packaging to use uv as the primary package manager while maintaining backward compatibility where needed. This includes migrating package definition to lib/pyproject.toml, consolidating tool configs in root pyproject.toml, and updating all Makefile targets and CI workflows to use `uv run` for proper virtual environment isolation. Co-Authored-By: Claude <noreply@anthropic.com>
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
✅ PR preview is ready!
|
There was a problem hiding this comment.
Pull request overview
This pull request migrates Streamlit's Python package management to use uv following the "Hybrid Approach" (Option C). The package definition moves from lib/setup.py to lib/pyproject.toml, tool configurations consolidate in the root pyproject.toml, and all development workflows are updated to use uv run for proper environment isolation.
Changes:
- Package definition migrated from
lib/setup.pytolib/pyproject.tomlwith proper PEP 517/518/621 compliance - Tool configurations (ruff, mypy, ty, pytest, coverage) consolidated from separate config files (
.ruff.toml,mypy.ini, etc.) topyproject.tomlfiles - Development/test/integration dependencies moved from
.txtfiles to PEP 735 dependency groups in rootpyproject.toml - All Makefile targets and CI workflows updated to use
uv syncanduv runcommands - Scripts updated to work with
pyproject.tomlinstead ofsetup.py
Reviewed changes
Copilot reviewed 34 out of 35 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
uv-migration.md |
Comprehensive migration plan documentation (new) |
pyproject.toml |
Root development environment config with tool configs and dependency groups (new) |
lib/pyproject.toml |
Package definition replacing setup.py (new) |
lib/setup.py |
Removed - replaced by pyproject.toml |
.ruff.toml |
Removed - migrated to pyproject.toml |
mypy.ini |
Removed - migrated to pyproject.toml |
ty.toml |
Removed - migrated to pyproject.toml |
lib/.coveragerc |
Removed - migrated to lib/pyproject.toml |
lib/dev-requirements.txt |
Removed - migrated to dependency groups |
lib/test-requirements.txt |
Removed - migrated to dependency groups |
lib/integration-requirements.txt |
Removed - migrated to dependency groups |
scripts/verify_version.py |
New script replacing setup.py verify command |
scripts/update_version.py |
Updated regex to match pyproject.toml version field |
scripts/update_name.py |
Updated regex to match pyproject.toml name field |
scripts/sync_ruff_version.py |
Updated to read ruff version from pyproject.toml |
Makefile |
All Python targets updated to use uv sync and uv run |
lib/conda-recipe/meta.yaml |
Updated to load version from pyproject.toml |
.github/workflows/* |
CI workflows updated for uv |
.github/dependabot.yml |
Changed package-ecosystem from pip to uv |
lib/tests/AGENTS.md |
Updated pytest commands to use uv run |
lib/AGENTS.md |
Updated config file references |
.gitignore |
Added uv.lock |
Comments suppressed due to low confidence (1)
Makefile:117
- Running
protocwithuv runis incorrect. Theprotoccommand is a system binary (the Protocol Buffer compiler), not a Python script that should be executed through uv. This will cause the protobuf compilation to fail because uv will try to findprotocas a Python package. The line should just beprotocwithout theuv runprefix.
uv run protoc \
--proto_path=proto \
--python_out=lib \
--mypy_out=lib \
proto/streamlit/proto/*.proto
- Change venv directory from 'venv' to '.venv' to align with uv's default - Update make_init action to use .venv for cache and activation - Update cache key version to v2 to invalidate old caches - Update all workflow files that reference venv/bin/activate - Fix mypy errors in scripts/verify_version.py - Simplify tomllib import logic - Add explicit type annotation to avoid Any return - Fix conda recipe to find pyproject.toml in correct location - Change path from 'pyproject.toml' to '../pyproject.toml' Co-Authored-By: Claude <noreply@anthropic.com>
|
@cursor review |
- Remove unused type: ignore comment from scripts/verify_version.py (tomli is a typed package, so type checkers find it correctly) - Remove pip check from conda recipe test section since Snowpark builds intentionally exclude some dependencies which would cause pip check to fail Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
- CONTRIBUTING.md: Update development setup instructions to use uv - Simplify Python environment section (uv handles venv automatically) - Update Python unit test commands to use `uv run pytest` - Update Streamlit run instructions to use `uv run` - Simplify troubleshooting section - Remove manual venv creation from Ubuntu setup - wiki/running-e2e-tests.md: Update pytest command to use `uv run` - .cursor/rules/new_feature.mdc: Update pytest command to use `uv run` Co-Authored-By: Claude <noreply@anthropic.com>
The cache key was not including the Python version, causing all jobs to share the same cached venv regardless of which Python version they needed. This resulted in errors like "No virtual environment found for Python 3.10" when the cache was created with Python 3.12. Fix: Include the Python version in the cache key to ensure each Python version gets its own cached virtual environment. Also bump version to v3 to force fresh caches. Co-Authored-By: Claude <noreply@anthropic.com>
📉 Frontend coverage change detectedThe frontend unit test (vitest) coverage has decreased by 0.0000%
✅ Coverage change is within normal range. |
SummaryThis PR migrates Streamlit's Python packaging infrastructure from the traditional
Code QualityStrengths
Minor Observations
Test CoveragePython Tests
No New E2E Tests RequiredThis PR is infrastructure/tooling focused and doesn't introduce new features or modify frontend behavior, so no new E2E tests are needed. Backwards CompatibilityMaintained Compatibility
Developer Environment Changes
Security & RiskLow Risk
Considerations
Recommendations
VerdictAPPROVED: This PR represents a well-executed migration to modern Python packaging with uv. The changes are comprehensive, consistent across all affected files, and maintain backward compatibility for end users. The CI improvements (caching validation, Ray test fixes) are thoughtful additions. The migration follows Python packaging best practices (PEP 517/518/621/735) and will improve the developer experience with faster dependency resolution and better reproducibility. This is an automated AI review. Please verify the feedback and use your judgment. |
- Add verify_version.py check to release.yml before packaging to ensure git tag matches pyproject.toml version (same as nightly.yml) - Restore asyncio_default_fixture_loop_scope = "function" to pytest config that was present in the old pytest.ini Co-Authored-By: Claude <noreply@anthropic.com>
SummaryThis PR migrates Streamlit’s Python tooling and CI to Code QualityThe migration is generally clean and consistent across Makefile, CI, and config. One potential regression stands out:
[project.scripts]
streamlit = "streamlit.web.cli:main"
[project.urls]
Homepage = "https://streamlit.io"
Documentation = "https://docs.streamlit.io/"
"Source Code" = "https://github.com/streamlit/streamlit"
"Bug Tracker" = "https://github.com/streamlit/streamlit/issues"
"Release Notes" = "https://docs.streamlit.io/develop/quick-reference/changelog"
Community = "https://discuss.streamlit.io/"
[tool.setuptools]
zip-safe = false
include-package-data = true
[tool.setuptools.packages.find]
exclude = ["tests", "tests.*"]
# PEP 561: https://mypy.readthedocs.io/en/stable/installed_packages.html
[tool.setuptools.package-data]
streamlit = ["py.typed", "hello/**/*.py"]Test Coverage
Backwards CompatibilityPotential compatibility risk on Windows if Security & RiskNo direct security concerns noted. Primary risk is Windows CLI regression if the Recommendations
VerdictCHANGES REQUESTED: Please confirm or restore Windows CLI script packaging to avoid a potential regression. This is an automated AI review using |
This change is intended since the new setup can handle this out-of-the box |
📉 Python coverage change detectedThe Python unit test coverage has decreased by 0.0302%
💡 Consider adding more unit tests to maintain or improve coverage. Coverage by files
|
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
| .venv | ||
| Pipfile.lock | ||
| Pipfile | ||
| uv.lock |
There was a problem hiding this comment.
In a future follow-up, we could consider adding the uv.lock file as well. But this might require some tweaks.
There was a problem hiding this comment.
How do you scan for dependency CVEs without including uv.lock?
Right now there is a high one for protobuf and it does not help us to look compliant.
There was a problem hiding this comment.
As a Python library, CVEs on Python dependencies are primarily relevant to safeguard the build process, and we have some tools, such as Snyk, to assist us with this. At install time, it's the responsibility of the app developer to take care of using up-to-date and safe dependency versions. The range of supported versions is deliberately kept very open, which is common for Python libraries. Nevertheless, I think it would be a good practices for us to have a checked in lock file for our dev/test dependencies. Just needs a bit of refactoring.
But its very different story for frontend dependencies since these are bundled and shipped in the Streamlit distribution and need to be kept safe and up-to-date from our side.
Describe your changes
Consolidates Streamlit's Python packaging to use uv as the primary package manager.
lib/pyproject.tomlpyproject.toml(ruff, mypy, ty, dependency groups)uv runfor proper venv isolationContribution License Agreement
By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.