10000 [MRG] BLD Specify build time dependencies via pyproject.toml by jeremiedbb · Pull Request #16244 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]

Skip to content

[MRG] BLD Specify build time dependencies via pyproject.toml #16244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 11, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ recursive-include sklearn *.c *.h *.pyx *.pxd *.pxi *.tp
recursive-include sklearn/datasets *.csv *.csv.gz *.rst *.jpg *.txt *.arff.gz *.json.gz
include COPYING
include README.rst
include pyproject.toml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed because we need this file in the sdist? Otherwise in the binary releases we don't need to have it. Is my understanding correct?

17 changes: 12 additions & 5 deletions build_tools/azure/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ elif [[ "$DISTRIB" == "conda-pip-latest" ]]; then
# conda is still used as a convenient way to install Python and pip.
make_conda "python=$PYTHON_VERSION"
python -m pip install -U pip
python -m pip install numpy scipy cython joblib
python -m pip install pytest==$PYTEST_VERSION pytest-cov pytest-xdist
python -m pip install pandas matplotlib pyamg
# do not install dependencies for lightgbm since it requires scikit-learn
Expand Down Expand Up @@ -120,7 +119,15 @@ except ImportError:
"
python -m pip list

# Use setup.py instead of `pip install -e .` to be able to pass the -j flag
# to speed-up the building multicore CI machines.
python setup.py build_ext --inplace -j 3
python setup.py develop
if [[ "$DISTRIB" == "conda-pip-latest" ]]; then
# Check that pip can automatically install the build dependencies from
# pyproject.toml using an isolated build environment:
pip install --verbose --editable .
else
# Use the pre-installed build dependencies and build directly in the
# current environment.
# Use setup.py instead of `pip install -e .` to be able to pass the -j flag
# to speed-up the building multicore CI machines.
python setup.py build_ext --inplace -j 3
python setup.py develop
fi
2 changes: 1 addition & 1 deletion build_tools/circle/build_test_pypy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export LOKY_MAX_CPU_COUNT="2"
export OMP_NUM_THREADS="1"

python setup.py build_ext --inplace -j 3
pip install -e .
pip install --no-build-isolation -e .

# Check that Python implementation is PyPy
python - << EOL
Expand Down
25 changes: 14 additions & 11 deletions doc/developers/advanced_installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ feature, code or documentation improvement).
#. Install Cython_ and build the project with pip in :ref:`editable_mode`::

pip install cython
pip install --verbose --editable .
pip install 10000 --verbose --no-build-isolation --editable .

#. Check that the installed scikit-learn has a version number ending with
`.dev0`::
Expand All @@ -71,8 +71,11 @@ feature, code or documentation improvement).

.. note::

You will have to re-run the ``pip install --editable .`` command every time
the source code of a Cython file is updated (ending in `.pyx` or `.pxd`).
You will have to run the ``pip install --no-build-isolation --editable .``
command every time the source code of a Cython file is updated
(ending in `.pyx` or `.pxd`). Use the ``--no-build-isolation`` flag to
avoid compiling the whole project each time, only the files you have
modified.

Dependencies
------------
Expand Down Expand Up @@ -152,9 +155,9 @@ Editable mode

If you run the development version, it is cumbersome to reinstall the package
each time you update the sources. Therefore it is recommended that you install
in with the ``pip install --editable .`` command, which allows you to edit the
code in-place. This builds the extension in place and creates a link to the
development directory (see `the pip docs
in with the ``pip install --no-build-isolation --editable .`` command, which
allows you to edit the code in-place. This builds the extension in place and
creates a link to the development directory (see `the pip docs
<https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs>`_).

This is fundamentally similar to using the command ``python setup.py develop``
Expand Down Expand Up @@ -207,7 +210,7 @@ environment variables in the current command prompt.

Finally, build scikit-learn from this command prompt::

pip install --verbose --editable .
pip install --verbose --no-build-isolation --editable .

.. _compiler_macos:

Expand Down Expand Up @@ -240,7 +243,7 @@ scikit-learn from source::
conda-forge::compilers conda-forge::llvm-openmp
conda activate sklearn-dev
make clean
pip install --verbose --editable .
pip install --verbose --no-build-isolation --editable .

.. note::

Expand Down Expand Up @@ -300,7 +303,7 @@ Finally, build scikit-learn in verbose mode (to check for the presence of the
``-fopenmp`` flag in the compiler commands)::

make clean
pip install --verbose --editable .
pip install --verbose --no-build-isolation --editable .

.. _compiler_linux:

Expand Down Expand Up @@ -349,7 +352,7 @@ in the user folder using conda::

conda create -n sklearn-dev numpy scipy joblib cython conda-forge::compilers
conda activate sklearn-dev
pip install --verbose --editable .
pip install --verbose --no-build-isolation --editable .

.. _compiler_freebsd:

Expand All @@ -372,7 +375,7 @@ can set the environment variables to these locations::

Finally, build the package using the standard command::

pip install --verbose --editable .
pip install --verbose --no-build-isolation --editable .

For the upcoming FreeBSD 12.1 and 11.3 versions, OpenMP will be included in
the base system and these steps will not be necessary.
Expand Down
11 changes: 8 additions & 3 deletions doc/developers/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ how to set up your git repository:

5. Install scikit-learn in editable mode::

$ pip install --editable .
$ pip install --no-build-isolation --editable .

for more details about advanced installation, see the
:ref:`install_bleeding_edge` section.
Expand Down Expand Up @@ -261,8 +261,13 @@ modifying code and submitting a PR:

.. note::

If you are modifying a Cython module, you have to re-run step 5 after modifications
and before testing them.
If you are modifying a Cython module, you have to re-compile after
modifications and before testing them::

pip install --no-build-isolation -e .

Use the ``--no-build-isolation`` flag to avoid compiling the whole project
each time, only the files you have modified.

It is often helpful to keep your local feature branch synchronized with the
latest changes of the main scikit-learn repository::
Expand Down
9 changes: 9 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[build-system]
# Minimum requirements for the build system to execute.
requires = [
"setuptools",
"wheel",
"Cython>=0.28.5",
"numpy>=1.13.3",
"scipy>=0.19.1",
]
0