gh-108828: Support selecting tests by labels #108829
Conversation
|
If possible, I would prefer to land my PR #108858 before. |
2b24913 to
5ae7161
Compare
There was a problem hiding this comment.
Thanks a lot for working on this, it is definetely a step in the right direction for our test suite.
I have a question about naming, though.
Some tests in Lib/test/test_regrtest.py would also be nice, or I can add them later :)
I already have several ideas on top of this PR!
| @@ -498,16 +500,34 @@ The :mod:`test.support` module defines the following functions: | |||
| rather than looking directly in the path directories. | |||
|
|
|||
|
|
|||
| .. function:: mark(label, *, globals=None) | |||
There was a problem hiding this comment.
I would prefer to always call them "marks". The main reason is that pytest uses the same idea and pytest's marks are well-known.
| .. function:: mark(label, *, globals=None) | |
| .. function:: mark(name, *, globals=None) |
Right now mark function adds a "label". It does not sound right.
There was a problem hiding this comment.
Or we can always call it "label".
Yes, it is similar to pytest's markers, but there are enough differences in applying them and filtering by them. I do not have preference, "label", "marker" and "tag" are all look like synonyms in this context to me.
| @@ -498,16 +500,34 @@ The :mod:`test.support` module defines the following functions: | |||
| rather than looking directly in the path directories. | |||
|
|
|||
|
|
|||
| .. function:: mark(label, *, globals=None) | |||
|
|
|||
| Add a label to tests. | |||
There was a problem hiding this comment.
| Add a label to tests. | |
| Add a mark to tests. |
Lib/test/libregrtest/main.py
Outdated
| @@ -78,6 +78,14 @@ def __init__(self, ns: Namespace): | |||
| self.ignore_tests: FilterTuple = tuple(ns.ignore_tests) | |||
| else: | |||
| self.ignore_tests = None | |||
| if ns.accept_labels: | |||
| self.accept_labels: tuple[str, ...] = tuple(ns.accept_labels) | |||
There was a problem hiding this comment.
For some reasons they should be immutable, as well as match_tests and ignore_tests.
In any case, I am planning to rewrite filtering by names. Instead of two tuples there will be more complex structure where order matters. And the same can be used for labels.
| @@ -1002,16 +1015,41 @@ def wrapper(self): | |||
| #======================================================================= | |||
| # unittest integration. | |||
|
|
|||
| def _id(obj): | |||
| return obj | |||
| def mark(label, *, globals=None): | |||
There was a problem hiding this comment.
I had a similar API in one of my OS libs, users didn't like it at all :(
I think it would be better to use mark(..., *, module=False)
This way we can:
- Simplify the API for readers (even if our users are people working on CPython)
- Use
framehack to get the neededglobals()
There was a problem hiding this comment.
I do not like to depend on such hack it tests. If it does not work, many unrelated tests will not even be able to load.
Since it is an internal API used in limited number of places, simplicity for users is less important. support.mark('gui', globals=globals()) is not much worse than support.mark('gui', module=True)
Uh oh!
There was an error while loading. Please reload this page.