8000 Merge f0fc433cb35c0866bca788360e7a0c669b5e5879 into 29d3d1fbafb6dc29b… · nipype/pydra@5955907 · GitHub
[go: up one dir, main page]
Skip to content

Commit 5955907

Browse files
adi611adi611
authored
Merge f0fc433 into 29d3d1f 
2 parents 29d3d1f + f0fc433 commit 5955907

File tree

3 files changed

+71
-4
lines changed

3 files changed

+71
-4
lines changed

‎.github/workflows/testdask.yml

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Dask Tests
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
pull_request:
8+
9+
jobs:
10+
test:
11+
name: Test
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
test-name:
16+
- test_boutiques
17+
- test_dockertask
18+
- test_graph
19+
- test_helpers
20+
- test_helpers_file
21+
- test_helpers_state
22+
- test_nipype1_convert
23+
- test_node_task
24+
- test_numpy_examples
25+
- test_profiles
26+
- test_shelltask
27+
- test_shelltask_inputspec
28+
- test_singularity
29+
- test_specs
30+
- test_state
31+
- test_submitter
32+
- test_task
33+
- test_tasks_files
34+
- test_workflow
35+
fail-fast: false
36+
37+
steps:
38+
- name: Set up Python 3.9
39+
uses: actions/setup-python@v2
40+
with:
41+
python-version: 3.9
42+
43+
- name: Install dependencies
44+
run: |
45+
python -m pip install --upgrade pip
46+
47+
- name: Checkout Pydra repo
48+
uses: actions/checkout@v3
49+
with:
50+
repository: ${{ github.repository }}
51+
52+
- name: Install pydra with Dask and test dependencies
53+
run: |
54+
pip install -e ".[test,dask]"
55+
56+
- name: Run tests
57+
run: |
58+
pytest -v --dask pydra/engine/tests/${{ matrix.test-name }}.py --cov pydra --cov-config .coveragerc --cov-report xml:cov.xml
59+
60+
- name: Upload to codecov
61+
run: codecov -f cov.xml -F unittests -e GITHUB_WORKFLOW

‎pydra/engine/tests/test_workflow.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4826,7 +4826,7 @@ def test_graph_5(tmpdir):
48264826
exporting_graphs(wf=wf, name=name)
48274827

48284828

4829-
@pytest.mark.timeout(20)
4829+
@pytest.mark.timeout(40)
48304830
def test_duplicate_input_on_split_wf(tmpdir):
48314831
"""checking if the workflow gets stuck if it has to run two tasks with equal checksum;
48324832
This can occur when splitting on a list containing duplicate values.
@@ -4852,7 +4852,7 @@ def printer(a):
48524852
assert res[0].output.out1 == "test" and res[1].output.out1 == "test"
48534853

48544854

4855-
@pytest.mark.timeout(40)
4855+
@pytest.mark.timeout(80)
48564856
def test_inner_outer_wf_duplicate(tmpdir):
48574857
"""checking if the execution gets stuck if there is an inner and outer workflows
48584858
that run two nodes with the exact same inputs.

‎pydra/engine/workers.py

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -879,8 +879,14 @@ async def exec_dask(self, runnable, rerun=False):
879879
from dask.distributed import Client
880880

881881
self.client = await Client(**self.client_args, asynchronous=True)
882-
future = self.client.submit(runnable._run, rerun)
883-
result = await future
882+
883+
if isinstance(runnable, TaskBase):
884+
future = self.client.submit(runnable._run, rerun)
885+
result = await future
886+
else: # it could be tuple that includes pickle files with tasks and inputs
887+
ind, task_main_pkl, task_orig = runnable
888+
future = self.client.submit(load_and_run, task_main_pkl, ind, rerun)
889+
result = await future
884890
return result
885891

886892
def close(self):

0 commit comments

Comments
 (0)
0