8000 Add GH action to render and open AWS marker report issue (#9131) · codeperl/localstack@d0836e8 · GitHub
[go: up one dir, main page]
Skip to content

Commit d0836e8

Browse files
dominikschubertdominikschubert
authored
Add GH action to render and open AWS marker report issue (localstack#9131)
1 parent adefae0 commit d0836e8

File tree

3 files changed

+253
-0
lines changed

3 files changed

+253
-0
lines changed

‎.github/bot_templates/MARKER_REPORT_ISSUE.md.j2

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
title: AWS Marker Report
3+
---
4+
# AWS Marker Report
5+
6+
- Repository: {{ data.meta.repo_url }}
7+
- Reference Commit: `{{ data.meta.commit_sha }}`
8+
- Timestamp: `{{ data.meta.timestamp }}`
9+
10+
This is an autogenerated report on our pytest marker usage with a special focus on our AWS compatibility markers, i.e. the ones prefixed with `aws_`.
11+
12+
## Overview
13+
14+
```text
15+
{% for name, count in data.aggregated.items() -%}
16+
{{ name }} : {{ count }}
17+
{% endfor -%}
18+
```
19+
20+
Both `aws_unknown` and `aws_needs_fixing` should be reduced to `0` over time.
21+
If you have some spare capacity please take one of these tests, try it against AWS and see if it works. Replace `aws_unknown` with the correct marker.
22+
To avoid the case where two people are concurrently working on one test case, please tick the box to "claim" a case when you want to work on it.
23+
24+
_Note_: The individual assignments here are based on the entries in the [CODEOWNERS]({{ data.meta.repo_url }}/blob/{{ data.meta.commit_sha }}/CODEOWNERS) file.
25+
26+
## unknown ({{ data.aggregated['aws_unknown'] }})
27+
28+
{% for item in data.owners_aws_unknown -%}
29+
- [ ] `{{ item.pytest_node_id }}` ([file]({{ item.file_url }})) {{ " ".join(item.owners) }}
30+
{% endfor %}
31+
32+
## needs_fixing ({{ data.aggregated['aws_needs_fixing'] }})
33+
34+
{% for item in data.owners_aws_needs_fixing -%}
35+
- [ ] `{{ item.pytest_node_id }}` ([file]({{ item.file_url }})) {{ " ".join(item.owners) }}
36+
{% endfor %}

‎.github/workflows/marker-report-issue.yml

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Open marker report GH issue
2+
on:
3+
# only manual for now
4+
workflow_dispatch:
5+
inputs:
6+
updateExistingIssue:
7+
description: 'Select the empty string "" to open duplicate issues, "true" to update duplicate issues and "false" to skip duplicate issues'
8+
required: false
9+
type: choice
10+
default: ''
11+
options:
12+
- ''
13+
- 'false'
14+
- 'true'
15+
16+
jobs:
17+
marker-report-issue:
18+
runs-on: ubuntu-latest
19+
timeout-minutes: 10
20+
steps:
21+
22+
- name: Checkout this repo
23+
uses: actions/checkout@v4
24+
with:
25+
fetch-depth: 0
26+
27+
- name: Set up Python
28+
id: setup-python
29+
uses: actions/setup-python@v4
30+
with:
31+
python-version: "3.10"
32+
33+
- name: Install dependencies
34+
run: make install-dev
35+
36+
# makes use of the marker report plugin localstack.testing.pytest.marker_report
37+
- name: Generate marker report
38+
env:
39+
PYTEST_ADDOPTS: "-p no:localstack.testing.pytest.fixtures -p no:localstack.testing.pytest.snapshot -p no:localstack.testing.pytest.filters -p no:localstack.testing.pytest.fixture_conflicts -p no:tests.fixtures -s --co --disable-warnings --marker-report --marker-report-path './target'"
40+
MARKER_REPORT_PROJECT_NAME: localstack
41+
MARKER_REPORT_COMMIT_SHA: ${{ github.sha }}
42+
run: |
43+
. ./.venv/bin/activate
44+
pip install codeowners
45+
python -m pytest tests/aws/
46+
mv ./target/marker-report*.json ./target/marker-report.json
47+
48+
- name: Enrich and render marker report
49+
env:
50+
MARKER_REPORT_PATH: ./target/marker-report.json
51+
CODEOWNERS_PATH: ./CODEOWNERS
52+
TEMPLATE_PATH: ./.github/bot_templates/MARKER_REPORT_ISSUE.md.j2
53+
OUTPUT_PATH: ./target/MARKER_REPORT_ISSUE.md
54+
GITHUB_REPO: ${{ github.repository }}
55+
COMMIT_SHA: ${{ github.sha }}
56+
run: |
57+
. ./.venv/bin/activate
58+
pip install codeowners
59+
python scripts/render_marker_report.py
60+
61+
- name: Print generated markdown
62+
run: |
63+
cat ./target/MARKER_REPORT_ISSUE.md
64+
65+
- name: Upload generated markdown
66+
uses: actions/upload-artifact@v3
67+
with:
68+
path: ./target/MARKER_REPORT_ISSUE.md
69+
70+
- name: Create GH issue from template
71+
uses: JasonEtco/create-an-issue@v2
72+
env:
73+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
74+
with:
75+
# `update_existing` actually has 3 possible values:
76+
# 1. not set => will always open duplicates
77+
# 2. false => will not update and will not open duplicates (NOOP if title conflict detected)
78+
# 3. true => will update an existing one if conflict detected
79+
update_existing: ${{ inputs.updateExistingIssue || '' }}
80+
# search_existing: open
81+
filename: ./target/MARKER_REPORT_ISSUE.md

‎scripts/render_marker_report.py

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
"""
2+
This script generates a markdown file with a summary of the current pytest marker usage,
3+
as well as a list of certain markers, their corresponding tests and the CODEOWNERs of these tests.
4+
5+
It takes a pytest marker report generated by localstack.testing.pytest.marker_report
6+
and extends it with data from the CODEOWNERS file.
7+
The resulting data is processed by using a jinja2 template for the resulting GH issue template.
8+
9+
10+
Example on how to run this script manually:
11+
12+
$ MARKER_REPORT_PATH='./target/marker-report.json' \
13+
CODEOWNERS_PATH=./CODEOWNERS \
14+
TEMPLATE_PATH=./.github/bot_templates/MARKER_REPORT_ISSUE.md.j2 \
15+
OUTPUT_PATH=./target/MARKER_REPORT_ISSUE.md \
16+
GITHUB_REPO=localstack/localstack \
17+
COMMIT_SHA=e62e04509d0f950af3027c0f6df4e18c7385c630 \
18+
python scripts/render_marker_report.py
19+
"""
20+
import dataclasses
21+
import datetime
22+
import json
23+
import os
24+
25+
import jinja2
26+
from codeowners import CodeOwners
27+
28+
29+
@dataclasses.dataclass
30+
class EnrichedReportMeta:
31+
timestamp: str
32+
repo_url: str
33+
commit_sha: str
34+
35+
36+
@dataclasses.dataclass
37+
class TestEntry:
38+
file_path: str
39+
pytest_node_id: str
40+
owners: list[str]
41+
file_url: str
42+
43+
44+
@dataclasses.dataclass
45+
class EnrichedReport:
46+
"""an object of this class is passed for template rendering"""
47+
48+
meta: EnrichedReportMeta
49+
aggregated: dict[str, int]
50+
owners_aws_unknown: list[TestEntry]
51+
owners_aws_needs_fixing: list[TestEntry]
52+
53+
54+
def load_file(filepath: str) -> str:
55+
with open(filepath, "r") as fd:
56+
return fd.read()
57+
58+
59+
def load_codeowners(codeowners_path):
60+
return CodeOwners(load_file(codeowners_path))
61+
62+
63+
def render_template(*, template: str, enriched_report: EnrichedReport) -> str:
64+
return jinja2.Template(source=template).render(data=enriched_report)
65+
66+
67+
def create_test_entry(entry, *, code_owners: CodeOwners, commit_sha: str, github_repo: str):
68+
rel_path = "".join(entry["file_path"].partition("tests/")[1:])
69+
return TestEntry(
70+
pytest_node_id=entry["node_id"],
71+
file_path=rel_path,
72+
owners=[o[1] for o in code_owners.of(rel_path)] or ["CODEOWNER_MISSING"],
73+
file_url=f"https://github.com/{github_repo}/blob/{commit_sha}/{rel_path}",
74+
)
75+
76+
77+
def enrich_with_codeowners(
78+
*, input_data: dict, github_repo: str, commit_sha: str, code_owners: CodeOwners
79+
) -> EnrichedReport:
80+
return EnrichedReport(
81+
meta=EnrichedReportMeta(
82+
timestamp=datetime.datetime.utcnow().isoformat(),
83+
repo_url=f"https://github.com/{github_repo}",
84+
commit_sha=commit_sha,
85+
),
86+
aggregated={
87+
k: v for k, v in input_data["aggregated_report"].items() if k.startswith("aws_")
88+
},
89+
owners_aws_unknown=sorted(
90+
[
91+
create_test_entry(
92+
e, code_owners=code_owners, github_repo=github_repo, commit_sha=commit_sha
93+
)
94+
for e in input_data["entries"]
95+
if "aws_unknown" in e["markers"]
96+
],
97+
key=lambda x: x.file_path,
98+
),
99+
owners_aws_needs_fixing=sorted(
100+
[
101+
create_test_entry(
102+
e, code_owners=code_owners, github_repo=github_repo, commit_sha=commit_sha
103+
)
104+
for e in input_data["entries"]
105+
if "aws_needs_fixing" in e["markers"]
106+
],
107+
key=lambda x: x.file_path,
108+
),
109+
)
110+
111+
112+
def main():
113+
marker_report_path = os.environ["MARKER_REPORT_PATH"]
114+
codeowners_path = os.environ["CODEOWNERS_PATH"]
115+
template_path = os.environ["TEMPLATE_PATH"]
116+
output_path = os.environ["OUTPUT_PATH"]
117+
github_repo = os.environ["GITHUB_REPO"]
118+
commit_sha = os.environ["COMMIT_SHA"]
119+
120+
code_owners = CodeOwners(load_file(codeowners_path))
121+
marker_report = json.loads(load_file(marker_report_path))
122+
enriched_report = enrich_with_codeowners(
123+
input_data=marker_report,
124+
github_repo=github_repo,
125+
commit_sha=commit_sha,
126+
code_owners=code_owners,
127+
)
128+
rendered_markdown = render_template(
129+
template=load_file(template_path), enriched_report=enriched_report
130+
)
131+
with open(output_path, "wt") as outfile:
132+
outfile.write(rendered_markdown)
133+
134+
135+
if __name__ == "__main__":
136+
main()

0 commit comments

Comments
 (0)
0