Fixture metadata available during collection / How do parametrized fixtures work? #14281

vladidobro · 2026-03-10T09:38:10Z

vladidobro
Mar 10, 2026

Hello,

I would appreciate a bit of guidance.

Context

I have a lot of data processing tests that I want to run against real data.
The data are in total potentially quite big (>100GB), so I don't have them checked out in the workdir, instead I store them in DVC (https://dvc.org/), but it could just as well be Git LFS.
I want to create a pytest plugin that transparently downloads the files on disk before running the tests.
There are also a lot of unrelated data files in my repo, so I don't want to download all the files upfront.

The simplest solution would be to download the files during fixture setup for each test, but that doesn't perform well. My files are in object storage and it is important to download them in parallel.

Ideally, I would like to

invoke pytest with some of its test selection mechanisms, like pytest tests/etl/ -m "dvc" -k processing
collect the tests that will be run
find out which files I need
download the files
run the tests

So I mark each test with the set of files it needs, and in a session-scope autouse fixture I collect needed files from each marked test and download it.

@mark.dvc(files=['data1.txt', 'data2.txt'])
def test_processing(): ...

All good.

But often these files are needed for some mock, e.g. for testing ETL pipelines

@fixture
def mock_client():
    # client = some requests mock
    # mount the files at some HTTP paths
    return client

def test_get()

in which case, I would have add the mark to each test using the fixture.
Ideally, if I select a test that uses the mock_client fixture, I would like to add the files needed for mock_client to the list of files that I need to download.

My question

Is there some way to add metadata to the fixtures that will be accessible at collection time?
I came across this thread #11462, where the verdict is "to track your metadata in some way unrelated to the pytest internals", which is very reasonable.

On the other hand, pytest supports parametrized fixtures

@fixture(params=[1, 2])

and these parameters show up in the collected tests, so apparently they must be available during collection!

How does this work? Does it use some mechanism that I could potentially also use?

Thank you very much for answers.

bluetech · 2026-03-11T11:34:51Z

bluetech
Mar 11, 2026
Maintainer

Quick idea which will mostly keep your current setup. Add a pytest_collection_finish hook to find all tests which request the fixtures you're interested in, then add the appropriate dvc mark to them. You can obtain the tests with session.items. To check which fixtures are requested by a test (item), you can use item.fixturenames. To add a marker, you can use item.add_marker(...).

There are various other solutions, but maybe this gets you going.

Is there some way to add metadata to the fixtures that will be accessible at collection time?

Currently, not really. Nothing built-in at least, I'm sure there's some plugin which does it :)

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fixture metadata available during collection / How do parametrized fixtures work? #14281

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Fixture metadata available during collection / How do parametrized fixtures work? #14281

Uh oh!

Uh oh!

vladidobro Mar 10, 2026

Context

My question

Replies: 1 comment

Uh oh!

bluetech Mar 11, 2026 Maintainer

vladidobro
Mar 10, 2026

bluetech
Mar 11, 2026
Maintainer