8000 Joblib 0.12.2 by ogrisel · Pull Request #11741 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Joblib 0.12.2 #11741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Aug 3, 2018
Merged

Joblib 0.12.2 #11741

merged 3 commits into from
Aug 3, 2018

Conversation

ogrisel
Copy link
Member
@ogrisel ogrisel commented Aug 3, 2018

Synchronize to the latest version of joblib.

  • includes loky: it fixes the intermittent crashes between OpenMP / Accelerate and GridSearchCV / RandomizedSearchCV;

  • makes it possible to better negotiate backends when using dask

  • better detection of the number of available CPUs and size of /dev/shm when running in containers such as docker

  • makes it possible to use LZ4 fast compression in load/dump

  • many other improvements and bug fixes.

@ogrisel ogrisel mentioned this pull request Aug 3, 2018
Closed
2 tasks
@ogrisel
Copy link
Member Author
ogrisel commented Aug 3, 2018

Ok this is good to go. Merging.

@ogrisel ogrisel merged commit 40e6c43 into scikit-learn:master Aug 3, 2018
@ogrisel ogrisel deleted the joblib-0.12.2 branch August 3, 2018 10:34
@qinhanmin2014
Copy link
Member

@ogrisel
(1) Circle CI is failing

Unexpected failing examples:
/home/circleci/project/examples/cluster/plot_feature_agglomeration_vs_univariate_selection.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/circleci/project/examples/cluster/plot_feature_agglomeration_vs_univariate_selection.py", line 75, in <module>
    clf.fit(X, y)  # set the best parameters
  File "/home/circleci/project/sklearn/model_selection/_search.py", line 663, in fit
    cv.split(X, y, groups)))
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 981, in __call__
    if self.dispatch_one_batch(iterator):
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 818, in dispatch_one_batch
    self._pickle_cache)
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 253, in __init__
    self.items = list(iterator_slice)
  File "/home/circleci/project/sklearn/model_selection/_search.py", line 662, in <genexpr>
    for parameters, (train, test) in product(candidate_params,
  File "/home/circleci/project/sklearn/base.py", line 62, in clone
    new_object_params[name] = clone(param, safe=False)
  File "/home/circleci/project/sklearn/base.py", line 50, in clone
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File "/home/circleci/project/sklearn/base.py", line 50, in <listcomp>
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File "/home/circleci/project/sklearn/base.py", line 50, in clone
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File "/home/circleci/project/sklearn/base.py", line 50, in <listcomp>
    return estimator_type([clone(e, safe=safe) for e in estimator])
  File "/home/circleci/project/sklearn/base.py", line 62, in clone
    new_object_params[name] = clone(param, safe=False)
  File "/home/circleci/project/sklearn/base.py", line 53, in clone
    return copy.deepcopy(estimator)
  File "/home/circleci/miniconda/envs/testenv/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/circleci/miniconda/envs/testenv/lib/python3.6/copy.py", line 274, in _reconstruct
    y = func(*args)
  File "/home/circleci/project/sklearn/externals/joblib/memory.py", line 830, in __init__
    location, cachedir))
ValueError: You set both "location='/tmp/tmp1v2mgts1' and "cachedir=False". 'cachedir' has been deprecated in version 0.12 and will be removed in version 0.14.
Please only set "location='/tmp/tmp1v2mgts1'"


/home/circleci/project/examples/compose/plot_compare_reduction.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/circleci/project/examples/compose/plot_compare_reduction.py", line 119, in <module>
    grid.fit(digits.data, digits.target)
  File "/home/circleci/project/sklearn/model_selection/_search.py", line 663, in fit
    cv.split(X, y, groups)))
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 981, in __call__
    if self.dispatch_one_batch(iterator):
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 818, in dispatch_one_batch
    self._pickle_cache)
  File "/home/circleci/project/sklearn/externals/joblib/parallel.py", line 253, in __init__
    self.items = list(iterator_slice)
  File "/home/circleci/project/sklearn/model_selection/_search.py", line 662, in <genexpr>
    for parameters, (train, test) in product(candidate_params,
  File "/home/circleci/project/sklearn/base.py", line 62, in clone
    new_object_params[name] = clone(param, safe=False)
  File "/home/circleci/project/sklearn/base.py", line 53, in clone
    return copy.deepcopy(estimator)
  File "/home/circleci/miniconda/envs/testenv/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/circleci/miniconda/envs/testenv/lib/python3.6/copy.py", line 274, in _reconstruct
    y = func(*args)
  File "/home/circleci/project/sklearn/externals/joblib/memory.py", line 830, in __init__
    location, cachedir))
ValueError: You set both "location='/tmp/tmpa408j1cz' and "cachedir=False". 'cachedir' has been deprecated in version 0.12 and will be removed in version 0.14.
Please only set "location='/tmp/tmpa408j1cz'"

(2) We get many extra warnings in the examples
e.g., UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = None.
(3) In the doc, we still have things like n_jobs : int, optional (default=1)
(4) What'e the difference between n_jobs=1 and n_jobs=None? Maybe add some explanations?

@js3711
Copy link
js3711 commented Aug 3, 2018

Thanks @ogrisel. Is there way to request a scikit-learn release candidate version through conda-forge (when it is ready)?

I'm excited about how dask will make use of these changes and I'd like to get a build for multiple platforms.

@jakirkham
Copy link
Contributor

We have built RCs for scikit-learn in the past. If scikit-learn plans to make an RC, we'd be happy to work with them to get it released. RCs for various packages are available via conda-forge/label/rc.

@jnothman
Copy link
Member
jnothman commented Aug 5, 2018 via email

@rth
Copy link
Member
rth commented Aug 13, 2018

Discussion about the reported issues continued in #11771

@qinhanmin2014 qinhanmin2014 mentioned this pull request Aug 13, 2018
Closed
@rth rth mentioned this pull request Aug 13, 2018
Merged
@qinhanmin2014 qinhanmin2014 mentioned this pull request Aug 15, 2018
Merged
@TomDLT TomDLT mentioned this pull request Oct 4, 2018
Closed
@jakirkham jakirkham mentioned this pull request Nov 3, 2018
Merged
This was referenced May 24, 2019
Closed
Merged
@jakirkham jakirkham mentioned this pull request Sep 9, 2019
Merged
@jrgriffiniii jrgriffiniii mentioned this pull request Jan 26, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ogrisel @qinhanmin2014 @js3711 @jakirkham @jnothman @rth
0