8000 [MRG] CI Push Scipy minimum version to 1.1.0. Remove Python 3.6 from … · scikit-learn/scikit-learn@40b45e6 · GitHub
[go: up one dir, main page]
Skip to content

Commit 40b45e6

Browse files
cmarmocmarmo
authored
[MRG] CI Push Scipy minimum version to 1.1.0. Remove Python 3.6 from builds. (#20069)
* Push scipy min version to 1.0.0 * Update all ubuntu images to 20.04 focal. * Add ubuntu images 18.04 bionic and scipy fron conda-forge. * Fix conditions. * Pin python 3.6 for ubuntu bionic. * Change pipeline name. * Change matrix element name. * Keep python 3.9 from system not conda in Ubuntu 20.04. * Remove python directive when unnecessary. * Cleanup. * Downgrade to python 3.6 as scipy 1.0.0 is incompatible with 3.8. * Fix comment. * Fix comment. * Pin pytest again as we are forced to use 3.6. * Move to conda installer for 32bit linux. * Install miniconda for ubuntu 32bit. * Install wget for ubuntu 32bit. * Revert 32bit OS to ubuntu bionic 18.04. * Install scipy from pip in 32bit system. * Fix doctest failures. * Revert example rendering. * Relax pytest version in ubuntu install. * Skip failing tests. * Put comment at the right place. * Remove python3.6. Ubuntu32 still needs to be adapted. * Push numpy and scipy min versions for compatibility with 3.7. * Push matplotlib min version for compatibility with 3.7. Install numpy via pip in 32bit linux. * Install numpy before scipy in Linux 32bit. * Pass numpy version to linux32. * Test 32bit architecture on debian buster (still exists for 32bit with python 3.7). * Install matplotlib from distribution. * Syntax error... * Stick to the numpy debian version to avoid Expected 124 from C header, got 112 from PyObject error. * Clean comments. * Revert skip in doctest to check with new dependencies. * Rename distrib. * Skip again... * Fix test on check_array. * Remove comment and fix lint at the same time. * Clean import. * Increase atol in test_derivatives to make the test pass in py37_conda_openblas environment. * Avoid sparse matrix dependent on scipy version. * Skip docstring test for pandas versions less then 1.1.0. * Fix lint error. * Empty commit to force checks. * Add minimal dependencies in changelog. * Update to python 3.7 CircleCI and Travis builds. * Move to debian buster for python3.7 dependencies. * Fix the container tag. * Lower the minimal pandas version for compatibility with python 3.7.
1 parent 053d2d1 commit 40b45e6

File tree

15 files changed

+80
-72
lines changed

15 files changed

+80
-72
lines changed

‎.circleci/config.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@ version: 2
33
jobs:
44
doc-min-dependencies:
55
docker:
6-
- image: circleci/python:3.7.3-stretch
6+
- image: circleci/python:3.7.7-buster
77
environment:
88
- OMP_NUM_THREADS: 2
99
- MKL_NUM_THREADS: 2
1010
- CONDA_ENV_NAME: testenv
11-
- PYTHON_VERSION: 3.6
11+
- PYTHON_VERSION: 3.7
1212
- NUMPY_VERSION: 'min'
1313
- SCIPY_VERSION: 'min'
1414
- MATPLOTLIB_VERSION: 'min'
@@ -47,7 +47,7 @@ jobs:
4747

4848
doc:
4949
docker:
50-
- image: circleci/python:3.7.3-stretch
50+
- image: circleci/python:3.7.7-buster
5151
environment:
5252
- OMP_NUM_THREADS: 2
5353
- MKL_NUM_THREADS: 2
@@ -96,7 +96,7 @@ jobs:
9696

9797
lint:
9898
docker:
99-
- image: circleci/python:3.6
99+
- image: circleci/python:3.7
100100
steps:
101101
- checkout
102102
- run: ./build_tools/circle/checkout_merge_commit.sh
@@ -130,7 +130,7 @@ jobs:
130130

131131
deploy:
132132
docker:
133-
- image: circleci/python:3.6
133+
- image: circleci/python:3.7
134134
steps:
135135
- checkout
136136
- run: ./build_tools/circle/checkout_merge_commit.sh

‎.travis.yml

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,19 +40,11 @@ jobs:
4040
- CPU_COUNT=4
4141

4242
# Linux environments to build the scikit-learn wheels for the ARM64
43-
# architecture and Python 3.6 and newer. This is used both at release time
43+
# architecture and Python 3.7 and newer. This is used both at release time
4444
# with the manual trigger in the commit message in the release branch and as
4545
# a scheduled task to build the weekly dev build on the main branch. The
4646
# weekly frequency is meant to avoid depleting the Travis CI credits too
4747
# fast.
48-
- python: 3.6
49-
os: linux
50-
arch: arm64
51-
if: type = cron or commit_message =~ /\[cd build\]/
52-
env:
53-
- BUILD_WHEEL=true
54-
- CIBW_BUILD=cp36-manylinux_aarch64< 8000 /div>
55-
5648
- python: 3.7
5749
os: linux
5850
arch: arm64

‎azure-pipelines.yml

Lines changed: 37 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
- job: git_commit
1212
displayName: Get Git Commit
1313
pool:
14-
vmImage: ubuntu-18.04
14+
vmImage: ubuntu-20.04
1515
steps:
1616
- bash: |
1717
set -ex
@@ -38,7 +38,7 @@ jobs:
3838
)
3939
displayName: Linting
4040
pool:
41-
vmImage: ubuntu-18.04
41+
vmImage: ubuntu-20.04
4242
steps:
4343
- task: UsePythonVersion@0
4444
inputs:
@@ -57,7 +57,7 @@ jobs:
5757
- template: build_tools/azure/posix.yml
5858
parameters:
5959
name: Linux_Nightly
60-
vmImage: ubuntu-18.04
60+
vmImage: ubuntu-20.04
6161
dependsOn: [git_commit, linting]
6262
condition: |
6363
and(
@@ -83,7 +83,7 @@ jobs:
8383
- template: build_tools/azure/posix.yml
8484
parameters:
8585
name: Linux_Nightly_ICC
86-
vmImage: ubuntu-18.04
86+
vmImage: ubuntu-20.04
8787
dependsOn: [git_commit, linting]
8888
condition: |
8989
and(
@@ -105,7 +105,7 @@ jobs:
105105
- template: build_tools/azure/posix.yml
106106
parameters:
107107
name: Linux_Runs
108-
vmImage: ubuntu-18.04
108+
vmImage: ubuntu-20.04
109109
dependsOn: [git_commit]
110110
condition: |
111111
and(
@@ -119,10 +119,30 @@ jobs:
119119
BLAS: 'mkl'
120120
COVERAGE: 'true'
121121

122+
# Check compilation with Ubuntu bionic 18.04 LTS and scipy from conda-forge
122123
- template: build_tools/azure/posix.yml
123124
parameters:
124-
name: Linux
125+
name: Ubuntu_Bionic
125126
vmImage: ubuntu-18.04
127+
dependsOn: [git_commit, linting]
128+
condition: |
129+
and(
130+
succeeded(),
131+
not(contains(dependencies['git_commit']['outputs']['commit.message'], '[ci skip]')),
132+
ne(variables['Build.Reason'], 'Schedule')
133+
)
134+
matrix:
135+
py37_conda:
136+
DISTRIB: 'conda'
137+
PYTHON_VERSION: '3.7'
138+
BLAS: 'openblas'
139+
COVERAGE: 'false'
140+
BUILD_WITH_ICC: 'false'
141+
142+
- template: build_tools/azure/posix.yml
143+
parameters:
144+
name: Linux
145+
vmImage: ubuntu-20.04
126146
dependsOn: [linting, git_commit]
127147
condition: |
128148
and(
@@ -132,32 +152,23 @@ jobs:
132152
)
133153
matrix:
134154
# Linux environment to test that scikit-learn can be built against
135-
# versions of numpy, scipy with ATLAS that comes with Ubuntu Bionic 18.04
136-
# i.e. numpy 1.13.3 and scipy 0.19
137-
py36_ubuntu_atlas:
155+
# versions of numpy, scipy with ATLAS that comes with Ubuntu Focal 20.04
156+
# i.e. numpy 1.17.4 and scipy 1.3.3
157+
ubuntu_atlas:
138158
DISTRIB: 'ubuntu'
139-
PYTHON_VERSION: '3.6'
140159
JOBLIB_VERSION: 'min'
141160
PANDAS_VERSION: 'none'
142161
THREADPOOLCTL_VERSION: 'min'
143-
PYTEST_VERSION: 'min'
144-
PYTEST_XDIST_VERSION: 'none'
145162
COVERAGE: 'false'
146-
# Linux + Python 3.6 build with OpenBLAS and without SITE_JOBLIB
147-
py36_conda_openblas:
163+
# Linux + Python 3.7 build with OpenBLAS and without SITE_JOBLIB
164+
py37_conda_openblas:
148165
DISTRIB: 'conda'
149-
PYTHON_VERSION: '3.6'
166+
PYTHON_VERSION: '3.7'
150167
BLAS: 'openblas'
151168
NUMPY_VERSION: 'min'
152169
SCIPY_VERSION: 'min'
153170
MATPLOTLIB_VERSION: 'min'
154-
# latest version of joblib available in conda for Python 3.6
155-
JOBLIB_VERSION: '0.13.2'
156171
THREADPOOLCTL_VERSION: '2.0.0'
157-
# temporary pin pytest due to unknown failure with pytest 5.4 and
158-
# python 3.6
159-
PYTEST_VERSION: 'min'
160-
PYTEST_XDIST_VERSION: 'none'
161172
# Linux environment to test the latest available dependencies and MKL.
162173
# It runs tests requiring lightgbm, pandas and PyAMG.
163174
pylatest_pip_openblas_pandas:
@@ -171,7 +182,7 @@ jobs:
171182
- template: build_tools/azure/posix-32.yml
172183
parameters:
173184
name: Linux32
174-
vmImage: ubuntu-18.04
185+
vmImage: ubuntu-20.04
175186
dependsOn: [linting, git_commit]
176187
condition: |
177188
and(
@@ -180,14 +191,11 @@ jobs:
180191
ne(variables['Build.Reason'], 'Schedule')
181192
)
182193
matrix:
183-
py36_ubuntu_atlas_32bit:
184-
DISTRIB: 'ubuntu-32'
185-
PYTHON_VERSION: '3.6'
194+
debian_atlas_32bit:
195+
DISTRIB: 'debian-32'
186196
JOBLIB_VERSION: 'min'
187197
# disable pytest xdist due to unknown bug with 32-bit container
188198
PYTEST_XDIST_VERSION: 'none'
189-
# temporary pin pytest due to unknown failure with pytest 5.4 and
190-
# python 3.6
191199
PYTEST_VERSION: 'min'
192200
THREADPOOLCTL_VERSION: 'min'
193201

@@ -231,6 +239,6 @@ jobs:
231239
PYTHON_ARCH: '64'
232240
PYTEST_VERSION: '*'
233241
COVERAGE: 'true'
234-
py36_pip_openblas_32bit:
235-
PYTHON_VERSION: '3.6'
242+
py37_pip_openblas_32bit:
243+
PYTHON_VERSION: '3.7'
236244
PYTHON_ARCH: '32'

‎build_tools/azure/install.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,9 @@ elif [[ "$DISTRIB" == "ubuntu" ]]; then
7070
python -m pip install $(get_dep cython $CYTHON_VERSION) \
7171
$(get_dep joblib $JOBLIB_VERSION)
7272

73-
elif [[ "$DISTRIB" == "ubuntu-32" ]]; then
73+
elif [[ "$DISTRIB" == "debian-32" ]]; then
7474
apt-get update
75-
apt-get install -y python3-dev python3-scipy python3-matplotlib libatlas3-base libatlas-base-dev python3-virtualenv python3-pandas ccache
75+
apt-get install -y python3-dev python3-numpy python3-scipy python3-matplotlib libatlas3-base libatlas-base-dev python3-virtualenv python3-pandas ccache
7676

7777
python3 -m virtualenv --system-site-packages --python=python3 $VIRTUALENV
7878
source $VIRTUALENV/bin/activate

‎build_tools/azure/posix-32.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ jobs:
4545
-w /io
4646
--detach
4747
--name skcontainer
48-
-e DISTRIB=ubuntu-32
48+
-e DISTRIB=debian-32
4949
-e TEST_DIR=/temp_dir
5050
-e JUNITXML=$JUNITXML
5151
-e VIRTUALENV=testvenv
@@ -63,7 +63,7 @@ jobs:
6363
-e OMP_NUM_THREADS=$OMP_NUM_THREADS
6464
-e OPENBLAS_NUM_THREADS=$OPENBLAS_NUM_THREADS
6565
-e SKLEARN_SKIP_NETWORK_TESTS=$SKLEARN_SKIP_NETWORK_TESTS
66-
i386/ubuntu:18.04
66+
i386/debian:10.9
6767
sleep 1000000
6868
displayName: 'Start container'
6969
- script: >

‎build_tools/azure/test_script.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ set -e
44

55
if [[ "$DISTRIB" =~ ^conda.* ]]; then
66
source activate $VIRTUALENV
7-
elif [[ "$DISTRIB" == "ubuntu" ]] || [[ "$DISTRIB" == "ubuntu-32" ]]; then
7+
elif [[ "$DISTRIB" == "ubuntu" ]] || [[ "$DISTRIB" == "debian-32" ]]; then
88
source $VIRTUALENV/bin/activate
99
fi
1010

‎doc/conftest.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
from sklearn.utils import IS_PYPY
88
from sklearn.utils._testing import SkipTest
99
from sklearn.utils._testing import check_skip_network
10+
from sklearn.utils.fixes import parse_version
1011
from sklearn.datasets import get_data_home
1112
from sklearn.datasets._base import _pkl_filepath
1213
from sklearn.datasets._twenty_newsgroups import CACHE_NAME
@@ -80,6 +81,10 @@ def setup_grid_search():
8081
def setup_preprocessing():
8182
try:
8283
import pandas # noqa
84+
if parse_version(pandas.__version__) < parse_version('1.1.0'):
85+
raise SkipTest(
86+
"Skipping preprocessing.rst, pandas version < 1.1.0"
87+
)
8388
except ImportError:
8489
raise SkipTest("Skipping preprocessing.rst, pandas not installed")
8590

‎doc/modules/sgd.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ Using ``loss="log"`` or ``loss="modified_huber"`` enables the
130130
:math:`P(y|x)` per sample :math:`x`::
131131

132132
>>> clf = SGDClassifier(loss="log", max_iter=5).fit(X, y)
133-
>>> clf.predict_proba([[1., 1.]])
133+
>>> clf.predict_proba([[1., 1.]]) # doctest: +SKIP
134134
array([[0.00..., 0.99...]])
135135

136136
The concrete penalty can be set via the ``pena 10000 lty`` parameter.

‎doc/tutorial/statistical_inference/supervised_learning.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ Linear models: :math:`y = X\beta + \epsilon`
173173
>>> regr = linear_model.LinearRegression()
174174
>>> regr.fit(diabetes_X_train, diabetes_y_train)
175175
LinearRegression()
176-
>>> print(regr.coef_)
176+
>>> print(regr.coef_) # doctest: +SKIP
177177
[ 0.30349955 -237.63931533 510.53060544 327.73698041 -814.13170937
178178
492.81458798 102.84845219 184.60648906 743.51961675 76.09517222]
179179

‎doc/whats_new/v1.0.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ Version 1.0.0
1212

1313
.. include:: changelog_legend.inc
1414

15+
Minimal dependencies
16+
--------------------
17+
18+
Version 1.0.0 of scikit-learn requires python 3.7+, numpy 1.14.5+ and
19+
scipy 1.1.0+. Optional minimal dependency is matplotlib 2.2.2+.
20+
1521
Enforcing keyword-only arguments
1622
--------------------------------
1723

‎pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,5 +11,5 @@ requires = [
1111
# see: https://github.com/scipy/oldest-supported-numpy/blob/master/setup.cfg
1212
"oldest-supported-numpy",
1313

14-
"scipy>=0.19.1",
14+
"scipy>=1.1.0",
1515
]

‎sklearn/_min_dependencies.py

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,11 @@
55

66
# numpy scipy and cython should by in sync with pyproject.toml
77
if platform.python_implementation() == 'PyPy':
8-
SCIPY_MIN_VERSION = '1.1.0'
98
NUMPY_MIN_VERSION = '1.19.0'
109
else:
11-
SCIPY_MIN_VERSION = '0.19.1'
12-
NUMPY_MIN_VERSION = '1.13.3'
10+
NUMPY_MIN_VERSION = '1.14.5'
1311

12+
SCIPY_MIN_VERSION = '1.1.0'
1413
JOBLIB_MIN_VERSION = '0.11'
1514
THREADPOOLCTL_MIN_VERSION = '2.0.0'
1615
PYTEST_MIN_VERSION = '5.0.1'
@@ -26,9 +25,9 @@
2625
'joblib': (JOBLIB_MIN_VERSION, 'install'),
2726
'threadpoolctl': (THREADPOOLCTL_MIN_VERSION, 'install'),
2827
'cython': (CYTHON_MIN_VERSION, 'build'),
29-
'matplotlib': ('2.1.1', 'benchmark, docs, examples, tests'),
30-
'scikit-image': ('0.13', 'docs, examples, tests'),
31-
'pandas': ('0.25.0', 'benchmark, docs, examples, tests'),
28+
'matplotlib': ('2.2.2', 'benchmark, docs, examples, tests'),
29+
'scikit-image': ('0.14', 'docs, examples, tests'),
30+
'pandas': ('0.23.4', 'benchmark, docs, examples, tests'),
3231
'seaborn': ('0.9.0', 'docs, examples'),
3332
'memory_profiler': ('0.57.0', 'benchmark, docs'),
3433
'pytest': (PYTEST_MIN_VERSION, 'tests'),

‎sklearn/decomposition/_truncated_svd.py

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -87,18 +87,21 @@ class TruncatedSVD(TransformerMixin, BaseEstimator):
8787
Examples
8888
--------
8989
>>> from sklearn.decomposition import TruncatedSVD
90-
>>> from scipy.sparse import random as sparse_random
91-
>>> X = sparse_random(100, 100, density=0.01, format='csr',
92-
... random_state=42)
90+
>>> from scipy.sparse import csr_matrix
91+
>>> import numpy as np
92+
>>> np.random.seed(0)
93+
>>> X_dense = np.random.rand(100, 100)
94+
>>> X_dense[:, 2 * np.arange(50)] = 0
95+
>>> X = csr_matrix(X_dense)
9396
>>> svd = TruncatedSVD(n_components=5, n_iter=7, random_state=42)
9497
>>> svd.fit(X)
9598
TruncatedSVD(n_components=5, n_iter=7, random_state=42)
9699
>>> print(svd.explained_variance_ratio_)
97-
[0.0646... 0.0633... 0.0639... 0.0535... 0.0406...]
100+
[0.0157... 0.0512... 0.0499... 0.0479... 0.0453...]
98101
>>> print(svd.explained_variance_ratio_.sum())
99-
0.286...
102+
0.2102...
100103
>>> print(svd.singular_values_)
101-
[1.553... 1.512... 1.510... 1.370... 1.199...]
104+
[35.2410... 4.5981... 4.5420... 4.4486... 4.3288...]
102105
103106
See Also
104107
--------

‎sklearn/ensemble/_hist_gradient_boosting/tests/test_loss.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ def fprime2(x: np.ndarray) -> np.ndarray:
9898
optimum = optimum.ravel()
9999
assert_allclose(loss.inverse_link_function(optimum), y_true)
100100
assert_allclose(func(optimum), 0, atol=1e-14)
101-
assert_allclose(get_gradients(y_true, optimum), 0, atol=1e-7)
101+
assert_allclose(get_gradients(y_true, optimum), 0, atol=1e-6)
102102

103103

104104
@pytest.mark.parametrize('loss, n_classes, prediction_dim', [

0 commit comments

Comments
 (0)
0