10000 Speed up and simplify github workflows by sdb9696 · Pull Request #1128 · python-kasa/python-kasa · GitHub
[go: up one dir, main page]
Skip to content

Speed up and simplify github workflows #1128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 27, 2024
Merged

Speed up and simplify github workflows #1128

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
188a0fc
Enable parallel tests with pytest-xdist
sdb9696 Sep 26, 2024
df2e324
Exclude pypy-windows
sdb9696 Sep 27, 2024
4a787a6
Use astral-sh/setup-uv github action
sdb9696 Sep 27, 2024
9db779b
Fix typo in publish.yaml
sdb9696 Sep 27, 2024
df301e6
Run all pre-commit checks in ci.yml
sdb9696 Sep 27, 2024
b9db613
Fix pre-commit check and use v3 setup-uv workflow action
sdb9696 Sep 27, 2024
371bbdb
Merge branch 'master' into janitor/parallel_tests
sdb9696 Sep 27, 2024
bbf1667
Tweak comment
sdb9696 Sep 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 13 additions & 49 deletions .github/actions/setup/action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
name: Setup Environment
description: Install requested pipx dependencies, configure the system python, and install uv and the package dependencies
description: Install uv, configure the system python, and the package dependencies

inputs:
uv-install-options:
default: ""
uv-version:
default: 0.4.5
default: 0.4.16
python-version:
required: true
cache-pre-commit:
Expand All @@ -17,66 +17,30 @@ inputs:
runs:
using: composite
steps:
- uses: "actions/setup-python@v5"
- name: Install uv
uses: astral-sh/setup-uv@v2
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This became an official astral action a few weeks ago and makes life so much easier that we don't have to do the pipx caching manually that we had to for poetry.


- name: "Setup python"
uses: "actions/setup-python@v5"
id: setup-python
with:
python-version: "${{ inputs.python-version }}"
allow-prereleases: true

- name: Setup pipx environment Variables
id: pipx-env-setup
# pipx default home and bin dir are not writable by the cache action
# so override them here and add the bin dir to PATH for later steps.
# This also ensures the pipx cache only contains uv
run: |
SEP="${{ !startsWith(runner.os, 'windows') && '/' || '\\' }}"
PIPX_CACHE="${{ github.workspace }}${SEP}pipx_cache"
echo "pipx-cache-path=${PIPX_CACHE}" >> $GITHUB_OUTPUT
echo "pipx-version=$(pipx --version)" >> $GITHUB_OUTPUT
echo "PIPX_HOME=${PIPX_CACHE}${SEP}home" >> $GITHUB_ENV
echo "PIPX_BIN_DIR=${PIPX_CACHE}${SEP}bin" >> $GITHUB_ENV
echo "PIPX_MAN_DIR=${PIPX_CACHE}${SEP}man" >> $GITHUB_ENV
echo "${PIPX_CACHE}${SEP}bin" >> $GITHUB_PATH
shell: bash

- name: Pipx cache
id: pipx-cache
uses: actions/cache@v4
with:
path: ${{ steps.pipx-env-setup.outputs.pipx-cache-path }}
key: cache-${{ inputs.cache-version }}-${{ runner.os }}-${{ runner.arch }}-python-${{ inputs.python-version }}-${{ steps.setup-python.outputs.python-version }}-pipx-${{ steps.pipx-env-setup.outputs.pipx-version }}-uv-${{ inputs.uv-version }}

- name: Install uv
if: steps.pipx-cache.outputs.cache-hit != 'true'
id: install-uv
shell: bash
run: |-
pipx install uv==${{ inputs.uv-version }} --python "${{ steps.setup-python.outputs.python-path }}"

- name: Read uv cache location
id: uv-cache-location
shell: bash
run: |-
echo "uv-cache-location=$(uv cache dir)" >> $GITHUB_OUTPUT

- uses: actions/cache@v4
name: uv cache
with:
path: |
${{ steps.uv-cache-location.outputs.uv-cache-location }}
key: cache-${{ inputs.cache-version }}-${{ runner.os }}-${{ runner.arch }}-python-${{ steps.setup-python.outputs.python-version }}-uv-${{ inputs.uv-version }}-${{ hashFiles('uv.lock') }}-options-${{ inputs.uv-install-options }}

- name: "uv install"
- name: "Install project"
shell: bash
run: |
uv sync --python "${{ steps.setup-python.outputs.python-path }}" ${{ inputs.uv-install-options }}
uv sync ${{ inputs.uv-install-options }}

- name: Read pre-commit version
if: inputs.cache-pre-commit == 'true'
id: pre-commit-version
shell: bash
run: >-
echo "pre-commit-version=$(uv run pre-commit -- -V | awk '{print $2}')" >> $GITHUB_OUTPUT
echo "pre-commit-version=$(uv run pre-commit -V | awk '{print $2}')" >> $GITHUB_OUTPUT

- uses: actions/cache@v4
if: inputs.cache-pre-commit == 'true'
Expand Down
43 changes: 15 additions & 28 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
workflow_dispatch: # to allow manual re-runs

env:
UV_VERSION: 0.4.5
UV_VERSION: 0.4.16

jobs:
linting:
Expand All @@ -20,39 +20,19 @@ jobs:
python-version: ["3.12"]

steps:
- uses: "actions/checkout@v4"
- name: "Checkout source files"
uses: "actions/checkout@v4"
- name: Setup environment
uses: ./.github/actions/setup
with:
python-version: ${{ matrix.python-version }}
cache-pre-commit: true
uv-version: ${{ env.UV_VERSION }}
uv-install-options: "--all-extras"
- name: "Check supported device md files are up to date"
run: |
uv run pre-commit run generate-supported --all-files
- name: "Linting and code formating (ruff)"
run: |
uv run pre-commit run ruff --all-files
- name: "Typing checks (mypy)"
run: |
source .venv/bin/activate
pre-commit run mypy --all-files
- name: "Run trailing-whitespace"
run: |
uv run pre-commit run trailing-whitespace --all-files
- name: "Run end-of-file-fixer"
run: |
uv run pre-commit run end-of-file-fixer --all-files
- name: "Run check-docstring-first"
run: |
uv run pre-commit run check-docstring-first --all-files
- name: "Run debug-statements"
run: |
uv run pre-commit run debug-statements --all-files
- name: "Run check-ast"

- name: "Run pre-commit checks"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we were missing the ruff format, check-yaml and doc8 from the CI. It's probably better to just run all the checks in a single step and pass --verbose to avoid having to keep them in sync.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, no need to repeat oneself here, I have been wanting to do this for a while 👍

run: |
uv run pre-commit run check-ast --all-files
pre-commit run --all-files --verbose


tests:
Expand Down Expand Up @@ -83,6 +63,13 @@ jobs:
- os: ubuntu-latest
python-version: "3.10"
extras: true
# Exclude pypy on windows due to significant performance issues
# running pytest
- os: windows-latest
python-version: "pypy-3.9"
- os: windows-latest
python-version: "pypy-3.10"


steps:
- uses: "actions/checkout@v4"
Expand All @@ -95,11 +82,11 @@ jobs:
- name: "Run tests (no coverage)"
if: ${{ startsWith(matrix.python-version, 'pypy') }}
run: |
uv run pytest
uv run pytest -n auto
- name: "Run tests (with coverage)"
if: ${{ !startsWith(matrix.python-version, 'pypy') }}
run: |
uv run pytest --cov kasa --cov-report xml
uv run pytest -n auto --cov kasa --cov-report xml
- name: "Upload coverage to Codecov"
if: ${{ !startsWith(matrix.python-version, 'pypy') }}
uses: "codecov/codecov-action@v4"
Expand Down
16 changes: 10 additions & 6 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ on:
types: [published]

env:
UV_VERSION: 0.4.5
UV_VERSION: 0.4.16
PYTHON_VERSION: 3.12

jobs:
build-n-publish:
Expand All @@ -14,16 +15,19 @@ jobs:
id-token: write

steps:
- uses: actions/checkout@master
- name: Checkout source files
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v2

- name: Setup python
uses: actions/setup-python@v4
with:
python-version: "3.x"
python-version: ${{ env.PYTHON_VERSION }}

- name: Install uv
run: |-
pipx install uv==${{ env.UV_VERSION }} --python "${{ steps.setup-python.outputs.python-path }}"
- name: Build a binary wheel and a source tarball
run: uv build

- name: Publish release on pypi
uses: pypa/gh-action-pypi-publish@release/v1
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ repos:

- repo: https://github.com/astral-sh/uv-pre-commit
# uv version.
rev: 0.4.5
rev: 0.4.16
hooks:
# Update the uv lockfile
- id: uv-lock
Expand Down
2 changes: 1 addition & 1 deletion kasa/tests/test_device.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def _get_subclasses(of_class):
and module.__package__ != "kasa.interfaces"
):
subclasses.add((module.__package__ + "." + name, obj))
return subclasses
return sorted(subclasses)


device_classes = pytest.mark.parametrize(
Expand Down
2 changes: 1 addition & 1 deletion kasa/tests/test_protocol.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ def _get_subclasses(of_class):
and name != "_deprecated_TPLinkSmartHomeProtocol"
):
subclasses.add((name, obj))
return subclasses
return sorted(subclasses)


@pytest.mark.parametrize(
Expand Down
4 changes: 3 additions & 1 deletion pyproject.toml
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ dev-dependencies = [
"coverage[toml]",
"pytest-timeout~=2.0",
"pytest-freezer~=0.4",
"mypy~=1.0"
"mypy~=1.0",
"pytest-xdist>=3.6.1",
]


Expand Down Expand Up @@ -105,6 +106,7 @@ markers = [
"requires_dummy: test requires dummy data to pass, skipped on real devices",
]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
timeout = 10

[tool.doc8]
Expand Down
24 changes: 24 additions & 0 deletions uv.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
0