8000 Speed up and simplify github workflows by sdb9696 · Pull Request #1128 · python-kasa/python-kasa · GitHub
[go: up one dir, main page]

Skip to content

Speed up and simplify github workflows #1128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 12 additions & 49 deletions .github/actions/setup/action.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
name: Setup Environment
description: Install requested pipx dependencies, configure the system python, and install uv and the package dependencies
description: Install uv, configure the system python, and the package dependencies

inputs:
uv-install-options:
default: ""
uv-version:
default: 0.4.5
default: 0.4.16
python-version:
required: true
cache-pre-commit:
Expand All @@ -17,66 +17,29 @@ inputs:
runs:
using: composite
steps:
- uses: "actions/setup-python@v5"
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

- name: "Setup python"
uses: "actions/setup-python@v5"
id: setup-python
with:
python-version: "${{ inputs.python-version }}"
allow-prereleases: true

- name: Setup pipx environment Variables
id: pipx-env-setup
# pipx default home and bin dir are not writable by the cache action
# so override them here and add the bin dir to PATH for later steps.
# This also ensures the pipx cache only contains uv
run: |
SEP="${{ !startsWith(runner.os, 'windows') && '/' || '\\' }}"
PIPX_CACHE="${{ github.workspace }}${SEP}pipx_cache"
echo "pipx-cache-path=${PIPX_CACHE}" >> $GITHUB_OUTPUT
echo "pipx-version=$(pipx --version)" >> $GITHUB_OUTPUT
echo "PIPX_HOME=${PIPX_CACHE}${SEP}home" >> $GITHUB_ENV
echo "PIPX_BIN_DIR=${PIPX_CACHE}${SEP}bin" >> $GITHUB_ENV
echo "PIPX_MAN_DIR=${PIPX_CACHE}${SEP}man" >> $GITHUB_ENV
echo "${PIPX_CACHE}${SEP}bin" >> $GITHUB_PATH
shell: bash

- name: Pipx cache
id: pipx-cache
uses: actions/cache@v4
with:
path: ${{ steps.pipx-env-setup.outputs.pipx-cache-path }}
key: cache-${{ inputs.cache-version }}-${{ runner.os }}-${{ runner.arch }}-python-${{ inputs.python-version }}-${{ steps.setup-python.outputs.python-version }}-pipx-${{ steps.pipx-env-setup.outputs.pipx-version }}-uv-${{ inputs.uv-version }}

- name: Install uv
if: steps.pipx-cache.outputs.cache-hit != 'true'
id: install-uv
shell: bash
run: |-
pipx install uv==${{ inputs.uv-version }} --python "${{ steps.setup-python.outputs.python-path }}"

- name: Read uv cache location
id: uv-cache-location
shell: bash
run: |-
echo "uv-cache-location=$(uv cache dir)" >> $GITHUB_OUTPUT

- uses: actions/cache@v4
name: uv cache
with:
path: |
${{ steps.uv-cache-location.outputs.uv-cache-location }}
key: cache-${{ inputs.cache-version }}-${{ runner.os }}-${{ runner.arch }}-python-${{ steps.setup-python.outputs.python-version }}-uv-${{ inputs.uv-version }}-${{ hashFiles('uv.lock') }}-options-${{ inputs.uv-install-options }}

- name: "uv install"
- name: "Install project"
shell: bash
run: |
uv sync --python "${{ steps.setup-python.outputs.python-path }}" ${{ inputs.uv-install-options }}
uv sync ${{ inputs.uv-install-options }}

- name: Read pre-commit version
if: inputs.cache-pre-commit == 'true'
id: pre-commit-version
shell: bash
run: >-
echo "pre-commit-version=$(uv run pre-commit -- -V | awk '{print $2}')" >> $GITHUB_OUTPUT
echo "pre-commit-version=$(uv run pre-commit -V | awk '{print $2}')" >> $GITHUB_OUTPUT

- uses: actions/cache@v4
if: inputs.cache-pre-commit == 'true'
Expand Down
43 changes: 15 additions & 28 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
workflow_dispatch: # to allow manual re-runs

env:
UV_VERSION: 0.4.5
UV_VERSION: 0.4.16

jobs:
linting:
Expand All @@ -20,39 +20,19 @@ jobs:
python-version: ["3.12"]

steps:
- uses: "actions/checkout@v4"
- name: "Checkout source files"
uses: "actions/checkout@v4"
- name: Setup environment
uses: ./.github/actions/setup
with:
python-version: ${{ matrix.python-version }}
cache-pre-commit: true
uv-version: ${{ env.UV_VERSION }}
uv-install-options: "--all-extras"
- name: "Check supported device md files are up to date"
run: |
uv run pre-commit run generate-supported --all-files
- name: "Linting and code formating (ruff)"
run: |
uv run pre-commit run ruff --all-files
- name: "Typing checks (mypy)"
run: |
source .venv/bin/activate
pre-commit run mypy --all-files
- name: "Run trailing-whitespace"
run: |
uv run pre-commit run trailing-whitespace --all-files
- name: "Run end-of-file-fixer"
run: |
uv run pre-commit run end-of-file-fixer --all-files
- name: "Run check-docstring-first"
run: |
uv run pre-commit run check-docstring-first --all-files
- name: "Run debug-statements"
run: |
uv run pre-commit run debug-statements --all-files
- name: "Run check-ast"

- name: "Run pre-commit checks"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we were missing the ruff format, check-yaml and doc8 from the CI. It's probably better to just run all the checks in a single step and pass --verbose to avoid having to keep them in sync.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, no need to repeat oneself here, I have been wanting to do this for a while 👍

run: |
uv run pre-commit run check-ast --all-files
uv run pre-commit run --all-files --verbose


tests:
Expand Down Expand Up @@ -83,6 +63,13 @@ jobs:
- os: ubuntu-latest
python-version: "3.10"
extras: true
# Exclude pypy on windows due to significant performance issues
# running pytest requires ~12 min instead of 2 min on other platforms
- os: windows-latest
python-version: "pypy-3.9"
- os: windows-latest
python-version: "pypy-3.10"


steps:
- uses: "actions/checkout@v4"
Expand All @@ -95,11 +82,11 @@ jobs:
- name: "Run tests (no coverage)"
if: ${{ startsWith(matrix.python-version, 'pypy') }}
run: |
uv run pytest
uv run pytest -n auto
- name: "Run tests (with coverage)"
if: ${{ !startsWith(matrix.python-version, 'pypy') }}
run: |
uv run pytest --cov kasa --cov-report xml
uv run pytest -n auto --cov kasa --cov-report xml
- name: "Upload coverage to Codecov"
if: ${{ !startsWith(matrix.python-version, 'pypy') }}
uses: "codecov/codecov-action@v4"
Expand Down
16 changes: 10 additions & 6 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ on:
types: [published]

env:
UV_VERSION: 0.4.5
UV_VERSION: 0.4.16
PYTHON_VERSION: 3.12

jobs:
build-n-publish:
Expand All @@ -14,16 +15,19 @@ jobs:
id-token: write

steps:
- uses: actions/checkout@master
- name: Checkout source files
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3

- name: Setup python
uses: actions/setup-python@v4
with:
python-version: "3.x"
python-version: ${{ env.PYTHON_VERSION }}

- name: Install uv
run: |-
pipx install uv==${{ env.UV_VERSION }} --python "${{ steps.setup-python.outputs.python-path }}"
- name: Build a binary wheel and a source tarball
run: uv build

- name: Publish release on pypi
uses: pypa/gh-action-pypi-publish@release/v1
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ repos:

- repo: https://github.com/astral-sh/uv-pre-commit
# uv version.
rev: 0.4.5
rev: 0.4.16
hooks:
# Update the uv lockfile
- id: uv-lock
Expand Down
2 changes: 1 addition & 1 deletion kasa/tests/test_device.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def _get_subclasses(of_class):
and module.__package__ != "kasa.interfaces"
):
subclasses.add((module.__package__ + "." + name, obj))
return subclasses
return sorted(subclasses)


device_classes = pytest.mark.parametrize(
Expand Down
2 changes: 1 addition & 1 deletion kasa/tests/test_protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ def _get_subclasses(of_class):
and name != "_deprecated_TPLinkSmartHomeProtocol"
):
subclasses.add((name, obj))
return subclasses
return sorted(subclasses)


@pytest.mark.parametrize(
Expand Down
4 changes: 3 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ dev-dependencies = [
"coverage[toml]",
"pytest-timeout~=2.0",
"pytest-freezer~=0.4",
"mypy~=1.0"
"mypy~=1.0",
"pytest-xdist>=3.6.1",
]


Expand Down Expand Up @@ -105,6 +106,7 @@ markers = [
"requires_dummy: test requires dummy data to pass, skipped on real devices",
]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
timeout = 10

[tool.doc8]
Expand Down
24 changes: 24 additions & 0 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
0