8000 Merge pull request #662 from tclose/type-coercion · nipype/pydra@6062274 · GitHub
[go: up one dir, main page]
Skip to content

Commit 6062274

Browse files
tclosetclose
authored
Merge pull request #662 from tclose/type-coercion
Merging now ahead of alpha release 🥳 (NB: I left out the relaxation to allow Parent (out) -> Child (in) to pass type-checking as it is a bit involved, and can wait until before we make a full release)
2 parents 9de0560 + 292fd3f commit 6062274

34 files changed

+4791
-2733
lines changed

‎.github/workflows/testslurm.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ jobs:
2121
docker pull $DOCKER_IMAGE
2222
# Have image running in background
2323
docker run `bash <(curl -s https://codecov.io/env)` -itd -h ernie --name slurm -v `pwd`:/pydra -e NO_ET=$NO_ET $DOCKER_IMAGE
24+
- name: Update python
25+
run: docker exec slurm bash -c "conda install python==3.8.15"
2426
- name: Display previous jobs with sacct
2527
run: |
2628
echo "Allowing ports/daemons time to start" && sleep 10

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ cov.xml
1515
.*.swp
1616
*~
1717
.idea
18+
*.venv
1819

1920
.DS_Store
2021

‎docs/components.rst

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -169,15 +169,17 @@ the Task execution, the user can set splitter and combiner attributes of the Sta
169169
.. code-block:: python
170170
171171
task_with_state =
172-
add2(x=[1, 5]).split("x").combine("x")
172+
add2().split(x=[1, 5]).combine("x")
173173
174174
In this example, the ``State`` class is responsible for creating a list of two
175175
separate inputs, *[{x: 1}, {x:5}]*, each run of the *Task* should get one
176-
element from the list.
177-
The results are grouped back when returning the result from the *Task*.
178-
While this example
179-
illustrates mapping and grouping of results over a single parameter, *Pydra*
180-
extends this to arbitrary combinations of input fields and downstream grouping
176+
element from the list. Note that in this case the value for `x` is set in the `split()`
177+
method, not at the task's initialisation.
178+
The `combine()` method, specifies that the results are grouped back when returning the
179+
result from the *Task*.
180+
181+
While this example illustrates mapping and grouping of results over a single parameter,
182+
*Pydra* extends this to arbitrary combinations of input fields and downstream grouping
181183
over nested dataflows. Details of how splitters and combiners power *Pydra*'s
182184
scalable dataflows are described in the next section.
183185

‎pydra/__init__.py

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -47,11 +47,3 @@ def check_latest_version():
4747

4848
if TaskBase._etelemetry_version_data is None:
4949
TaskBase._etelemetry_version_data = check_latest_version()
50-
51-
52-
# attr run_validators is set to False, but could be changed using use_validator
53-
attr.set_run_validators(False)
54-
55-
56-
def set_input_validator(flag=False):
57-
attr.set_run_validators(flag)

‎pydra/engine/audit.py

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,10 @@
33
import json
44
import attr
55
from ..utils.messenger import send_message, make_message, gen_uuid, now, AuditFlag
6-
from .helpers import ensure_list, gather_runtime_info, hash_file
7-
from .specs import attr_fields, File, Directory
6+
from ..utils.hash import hash_function
7+
from .helpers import ensure_list, gather_runtime_info
8+
from .specs import attr_fields
9+
from fileformats.core import FileSet
810

911
try:
1012
import importlib_resources
@@ -181,10 +183,11 @@ def audit_task(self, task):
181183
command = task.cmdline if hasattr(task.inputs, "executable") else None
182184
attr_list = attr_fields(task.inputs)
183185
for attrs in attr_list:
184-
if attrs.type in [File, Directory]:
185-
input_name = attrs.name
186-
input_path = os.path.abspath(getattr(task.inputs, input_name))
187-
file_hash = hash_file(input_path)
186+
input_name = attrs.name
187+
value = getattr(task.inputs, input_name)
188+
if isinstance(value, FileSet):
189+
input_path = os.path.abspath(value)
190+
file_hash = hash_function(value)
188191
entity_id = f"uid:{gen_uuid()}"
189192
entity_message = {
190193
"@id": entity_id,

0 commit comments

Comments
 (0)
0