8000 [MRG] Ignore PendingDepWarnings of matrix subclass with pytest by NicolasHug · Pull Request #13076 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

[MRG] Ignore PendingDepWarnings of matrix subclass with pytest #13076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 2, 2019
Merged

[MRG] Ignore PendingDepWarnings of matrix subclass with pytest #13076

merged 1 commit into from
Feb 2, 2019

Conversation

NicolasHug
Copy link
Member
@NicolasHug NicolasHug commented Feb 1, 2019 •
edited by rth
Loading

Reference Issues/PRs

Partially addresses #12327

What does this implement/fix? Explain your changes.

numpy is deprecating the use of the matrix class.
scipy still uses it extensively, and has filtered out the related pending deprecation warnings in scipy/scipy#8887.

However these warnings still s 8000 how up (for a reason that I don't understand) in pytest (scipy/scipy#9734).

This PR makes pytest ignore these warnings and avoids incredibly long logs in appveyor.

Any other comments?

qinhanmin2014
qinhanmin2014 approved these changes Feb 2, 2019
View reviewed changes
Copy link
Member
@qinhanmin2014 qinhanmin2014 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the log is much more readable now :)

rth
rth approved these changes Feb 2, 2019
View reviewed changes
Copy link
Member
@rth rth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NicolasHug !

Closes #12327

I would rather still keep that issue open, as any project depending on scikit-learn will still have this issue.

@rth rth merged commit b792382 into scikit-learn:master Feb 2, 2019
@rth rth mentioned this pull request Feb 2, 2019
Closed
thomasjpfan pushed a commit to thomasjpfan/scikit-learn that referenced this pull request Feb 6, 2019
@NicolasHug @thomasjpfan
MNT Ignore PendingDeprecationWarnings of matrix subclass with pytest (s…
0c851e4 
…cikit-learn#13076)
thomasjpfan pushed a commit to thomasjpfan/scikit-learn that referenced this pull request Feb 7, 2019
@NicolasHug @thomasjpfan
MNT Ignore PendingDeprecationWarnings of matrix subclass with pytest (s…
14fe318 
…cikit-learn#13076)
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
@NicolasHug
MNT Ignore PendingDeprecationWarnings of matrix subclass with pytest (s…
a82f146 
…cikit-learn#13076)
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
Revert "MNT Ignore PendingDeprecationWarnings of matrix subclass with…
680d053 
… pytest (scikit-learn#13076)"

This reverts commit a82f146.
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
Revert "MNT Ignore PendingDeprecationWarnings of matrix subclass with…
c5e9684 
… pytest (scikit-learn#13076)"

This reverts commit a82f146.
koenvandevelde pushed a commit to koenvandevelde/scikit-learn that referenced this pull request Jul 12, 2019
@NicolasHug @koenvandevelde
MNT Ignore PendingDeprecationWarnings of matrix subclass with pytest (s…
e040873 
…cikit-learn#13076)
johnyf added a commit to tulip-control/tulip-control that referenced this pull request Jun 10, 2021
@johnyf
BUG: use numpy.ndarray, instead of numpy.matrix
47d1e5b 
because matrices have been deprecated in `numpy`. Usage of `scipy.sparse` is
causing this issue when conversions to matrices are performed. I changed the:
- calls to the method `scipy.sparse.lil.lil_matrix.todense`
- to calls to the method `scipy.sparse.lil.lil_matrix.toarray`,

to avoid the conversions that raise `PendingDeprecationWarning`s. The warnings
are raised by the call to the method `lil_matrix.todense` because this call
involves the instantiation of the class `numpy.matrix`, which is deprecated.
In contrast, the method `lil_matrix.toarray` creates an instance of
`numpy.ndarray`.

(In fact, in the module `tulip.abstract.discretization`, wherever the method
`lil_matrix.todense` was called, the value that it returned was immediately
converted to a `numpy.ndarray`. So calling the method `lil_matrix.toarray` is
actually more efficient.)

The class `numpy.matrix` is deprecated and will probably be removed in
the future. This will happen after arrangements have been made for
`scipy.space`. (For these points and more information, read the references
listed at the end.)

Still, I do not think that continuing to use `scipy.sparse.lil_matrix` until
when `numpy` removes matrices is a safe approach.
Instead, using `numpy.ndarray` would be safer.
Moreover, I do think that there are other data structures that would fit
this use case better than sparse matrices.


## Diagnosis

I describe below the approach I (eventually) followed to debug this warning,
because finding the cause was difficult.

The issue is a `PendingDeprecationWarning` issued from `numpy`.
This warning is visible in `pytest` runs, but *not* when running the Python
test file directly. Moreover, `pytest` reports the warning, and from which
test function it originates. The warning itself reads (I have wrapped
the lines here):

```
===================================== warnings summary ======================================
abstract_test.py::transition_directions_test
  /.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69:
  PendingDeprecationWarning: the matrix subclass is not the recommended way to
  represent matrices or deal with linear algebra
  (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html).
  Please adjust your code to use regular ndarray.
    return matrix(data, dtype=dtype, copy=False)
```

So the line in `tulip` that causes the warning cannot be found from the
information contained in the warning.

The above `PendingDeprecationWarning` was introduced in `numpy` in commit:
    numpy/numpy@11e9d2a
This warning was then ignored in the module `scipy.sparse.__init__`, in `scipy` commit:
    scipy/scipy@a874bd5
It appears that this configuration of warnings by `scipy` interacts with
`pytest` complexly:

- when running with `pytest abstract_test.py`,
  the `PendingDeprecationWarning` is visible, but

- when running with `python -X dev -- abstract_test.py`,
  the `PendingDeprecationWarning` is invisible. This behavior is due to
  the call:

  ```python
  warnings.filterwarnings(
      'ignore',
      message='the matrix subclass is not the recommended way')
  ```

  within `scipy.sparse.__init__.py` (introduced in the `scipy` commit
  that was mentioned above). Read also:
  https://docs.python.org/3/library/exceptions.html#PendingDeprecationWarning
  https://www.python.org/dev/peps/pep-0565/

As a result, it is difficult to find the cause within `tulip` of this
`PendingDeprecationWarning`.


## Getting a traceback

The test function that triggered the `PendingDeprecationWarning` from `numpy`
was not failing, so there was no traceback that would indicate which line in
`tulip` caused the warning.

In addition, there was an earlier warning issued by `matplotlib`. So turning
warnings to errors with the argument `-Werror` would cause `pytest` to turn the
`matplotlib` warning into an error, and stop before the warning of interest:

```shell
pytest -Werror abstract_test.py
```

So first I removed the `matplotlib` warnings (temporarily), by commenting the
line `matplotlib.use('Agg')` in the file `abstract_test.py`. This made the
warning of interest to become the first warning. I then passed `-Werror` to
`pytest`, and this turned the `numpy` warning into an error, which produced
the traceback shown below:

(The cause of these `matplotlib` warnings (there are two) is in the package
`polytope`, and has been addressed there, in commit:
    tulip-control/polytope@c464818
These changes will become available to `tulip` with the next `polytope` release.
Until then, the CI tests of `tulip` will raise these `matplotlib` warnings.
These warnings could be explicitly ignored by using `with pytest.warns`.)

(The paths to `tulip` in the traceback below lead to the repository's `tulip`,
instead of a directory under Python's `site-packages`, because during this
phase of debugging I installed `tulip` with `pip install -e .`, to iterate
faster while debugging.)

```
../tulip/abstract/discretization.py:1666: in discretize_switched
    plot_mode_partitions(merged_abstr, show_ts, only_adjacent)
../tulip/abstract/discretization.py:1673: in plot_mode_partitions
    axs = swab.plot(show_ts, only_adjacent)
../tulip/abstract/discretization.py:187: in plot
    ax = ab.plot(show_ts, only_adjacent, color_seed)
../tulip/abstract/discretization.py:403: in plot
    ax = _plot_abstraction(self, show_ts, only_adjacent,
../tulip/abstract/discretization.py:446: in _plot_abstraction
    ax = ab.ppp.plot(
../tulip/abstract/prop2partition.py:600: in plot
    return plot_partition(
.../.virtualenvs/.../lib/python3.9/site-packages/polytope/plot.py:90: in plot_partition
    trans = nx.to_numpy_matrix(trans, nodelist=ppp2trans)
.../.virtualenvs/.../lib/python3.9/site-packages/networkx/convert_matrix.py:553: in to_numpy_matrix
    M = np.asmatrix(A, dtype=dtype)
.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69: in asmatrix
    return matrix(data, dtype=dtype, copy=False)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

subtype = <class 'numpy.matrix'>
data = array([[1., 0., 0., 0., 0., 0.],
       [0., 1., 1., 0., 1., 1.],
       [1., 0., 1., 1., 0., 1.],
       [1., 0., 0., 1., 0., 0.],
       [0., 0., 0., 0., 1., 1.],
       [1., 0., 0., 0., 0., 1.]])
dtype = None, copy = False

    def __new__(subtype, data, dtype=None, copy=True):
>       warnings.warn('the matrix subclass is not the recommended way to '
                      'represent matrices or deal with linear algebra (see '
                      'https://docs.scipy.org/doc/numpy/user/'
                      'numpy-for-matlab-users.html). '
                      'Please adjust your code to use regular ndarray.',
                      PendingDeprecationWarning, stacklevel=2)
E       PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.

.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:116: PendingDeprecationWarning
```

As the traceback shows, the issue is due to a call to the function
`networkx.to_numpy_matrix` within the function `polytope.plot.plot_partition`.
So avoiding this warning will be possible after the next release of the
package `polytope`.

(Note that inserting an `assert False` in a suitable line within the
function `transition_directions_test` is not an alternative to passing the
argument `-Werror`, because the `assert False` will result in a traceback
where the `assert` statement appears, instead of a traceback that shows the
call stack at the point where the warning was issued.)


## Speeding up debugging using `pytest`

Also, since I had to be running `pytest` on the Python file `abstract_test.py`,
`pytest` would collect all test functions, and run them. The file
`abstract_test.py` happens to contain several slow test functions, so running
them all just to observe the results for the one function of interest is not
time-efficient.

What I did to speed up runs was to rename all `test_*` functions contained in
`abstract_test.py`, except for the one function of interest (namely
`transition_directions_test`), to identifiers outside the patterns collected
by `pytest`.

A simpler alternative, for use with larger test files, is to do the opposite:
rename only the function of interest to a different pattern, and then change
the line `python_functions = ` in t
8000
he configuration file `pytest.ini`.


## References

- numpy/numpy#10142  (DEP: Pending deprecation warning for matrix)
- numpy/numpy#10973  (DOC: advise against use of matrix)
- scipy/scipy#8887  (MAINT: filter out np.matrix PendingDeprecationWarning's in numpy >=1.15)
- scipy/scipy#9734  (PendingDeprecationWarning for np.matrix with pytest)
- scikit-learn/scikit-learn#12327  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices)
- scikit-learn/scikit-learn#13076  ([MRG] Ignore PendingDepWarnings of matrix subclass with pytest)
- cvxpy/cvxpy#567  (NumPy matrix class is pending deprecation and issuing warnings)
- cvxpy/cvxpy#637  (RF: Use a 2D np array instead of matrix to represent scalars)
- cvxpy/cvxpy#638  (RF: Change np.matrix to np.array in several places)
- cvxpy/cvxpy#644  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra)
- https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning
johnyf added a commit to tulip-control/tulip-control that referenced this pull request Jun 20, 2021
@johnyf
BUG: use numpy.ndarray, instead of numpy.matrix
93966e3 
because matrices have been deprecated in `numpy`. Usage of `scipy.sparse` is
causing this issue when conversions to matrices are performed. I changed the:
- calls to the method `scipy.sparse.lil.lil_matrix.todense`
- to calls to the method `scipy.sparse.lil.lil_matrix.toarray`,

to avoid the conversions that raise `PendingDeprecationWarning`s. The warnings
are raised by the call to the method `lil_matrix.todense` because this call
involves the instantiation of the class `numpy.matrix`, which is deprecated.
In contrast, the method `lil_matrix.toarray` creates an instance of
`numpy.ndarray`.

(In fact, in the module `tulip.abstract.discretization`, wherever the method
`lil_matrix.todense` was called, the value that it returned was immediately
converted to a `numpy.ndarray`. So calling the method `lil_matrix.toarray` is
actually more efficient.)

The class `numpy.matrix` is deprecated and will probably be removed in
the future. This will happen after arrangements have been made for
`scipy.space`. (For these points and more information, read the references
listed at the end.)

Still, I do not think that continuing to use `scipy.sparse.lil_matrix` until
when `numpy` removes matrices is a safe approach.
Instead, using `numpy.ndarray` would be safer.
Moreover, I do think that there are other data structures that would fit
this use case better than sparse matrices.


## Diagnosis

I describe below the approach I (eventually) followed to debug this warning,
because finding the cause was difficult.

The issue is a `PendingDeprecationWarning` issued from `numpy`.
This warning is visible in `pytest` runs, but *not* when running the Python
test file directly. Moreover, `pytest` reports the warning, and from which
test function it originates. The warning itself reads (I have wrapped
the lines here):

```
===================================== warnings summary ======================================
abstract_test.py::transition_directions_test
  /.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69:
  PendingDeprecationWarning: the matrix subclass is not the recommended way to
  represent matrices or deal with linear algebra
  (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html).
  Please adjust your code to use regular ndarray.
    return matrix(data, dtype=dtype, copy=False)
```

So the line in `tulip` that causes the warning cannot be found from the
information contained in the warning.

The above `PendingDeprecationWarning` was introduced in `numpy` in commit:
    numpy/numpy@11e9d2a
This warning was then ignored in the module `scipy.sparse.__init__`, in `scipy` commit:
    scipy/scipy@a874bd5
It appears that this configuration of warnings by `scipy` interacts with
`pytest` complexly:

- when running with `pytest abstract_test.py`,
  the `PendingDeprecationWarning` is visible, but

- when running with `python -X dev -- abstract_test.py`,
  the `PendingDeprecationWarning` is invisible. This behavior is due to
  the call:

  ```python
  warnings.filterwarnings(
      'ignore',
      message='the matrix subclass is not the recommended way')
  ```

  within `scipy.sparse.__init__.py` (introduced in the `scipy` commit
  that was mentioned above). Read also:
  https://docs.python.org/3/library/exceptions.html#PendingDeprecationWarning
  https://www.python.org/dev/peps/pep-0565/

As a result, it is difficult to find the cause within `tulip` of this
`PendingDeprecationWarning`.


## Getting a traceback

The test function that triggered the `PendingDeprecationWarning` from `numpy`
was not failing, so there was no traceback that would indicate which line in
`tulip` caused the warning.

In addition, there was an earlier warning issued by `matplotlib`. So turning
warnings to errors with the argument `-Werror` would cause `pytest` to turn the
`matplotlib` warning into an error, and stop before the warning of interest:

```shell
pytest -Werror abstract_test.py
```

So first I removed the `matplotlib` warnings (temporarily), by commenting the
line `matplotlib.use('Agg')` in the file `abstract_test.py`. This made the
warning of interest to become the first warning. I then passed `-Werror` to
`pytest`, and this turned the `numpy` warning into an error, which produced
the traceback shown below:

(The cause of these `matplotlib` warnings (there are two) is in the package
`polytope`, and has been addressed there, in commit:
    tulip-control/polytope@c464818
These changes will become available to `tulip` with the next `polytope` release.
Until then, the CI tests of `tulip` will raise these `matplotlib` warnings.
These warnings could be explicitly ignored by using `with pytest.warns`.)

(The paths to `tulip` in the traceback below lead to the repository's `tulip`,
instead of a directory under Python's `site-packages`, because during this
phase of debugging I installed `tulip` with `pip install -e .`, to iterate
faster while debugging.)

```
../tulip/abstract/discretization.py:1666: in discretize_switched
    plot_mode_partitions(merged_abstr, show_ts, only_adjacent)
../tulip/abstract/discretization.py:1673: in plot_mode_partitions
    axs = swab.plot(show_ts, only_adjacent)
../tulip/abstract/discretization.py:187: in plot
    ax = ab.plot(show_ts, only_adjacent, color_seed)
../tulip/abstract/discretization.py:403: in plot
    ax = _plot_abstraction(self, show_ts, only_adjacent,
../tulip/abstract/discretization.py:446: in _plot_abstraction
    ax = ab.ppp.plot(
../tulip/abstract/prop2partition.py:600: in plot
    return plot_partition(
.../.virtualenvs/.../lib/python3.9/site-packages/polytope/plot.py:90: in plot_partition
    trans = nx.to_numpy_matrix(trans, nodelist=ppp2trans)
.../.virtualenvs/.../lib/python3.9/site-packages/networkx/convert_matrix.py:553: in to_numpy_matrix
    M = np.asmatrix(A, dtype=dtype)
.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69: in asmatrix
    return matrix(data, dtype=dtype, copy=False)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

subtype = <class 'numpy.matrix'>
data = array([[1., 0., 0., 0., 0., 0.],
       [0., 1., 1., 0., 1., 1.],
       [1., 0., 1., 1., 0., 1.],
       [1., 0., 0., 1., 0., 0.],
       [0., 0., 0., 0., 1., 1.],
       [1., 0., 0., 0., 0., 1.]])
dtype = None, copy = False

    def __new__(subtype, data, dtype=None, copy=True):
>       warnings.warn('the matrix subclass is not the recommended way to '
                      'represent matrices or deal with linear algebra (see '
                      'https://docs.scipy.org/doc/numpy/user/'
                      'numpy-for-matlab-users.html). '
                      'Please adjust your code to use regular ndarray.',
                      PendingDeprecationWarning, stacklevel=2)
E       PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.

.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:116: PendingDeprecationWarning
```

As the traceback shows, the issue is due to a call to the function
`networkx.to_numpy_matrix` within the function `polytope.plot.plot_partition`.
So avoiding this warning will be possible after the next release of the
package `polytope`.

(Note that inserting an `assert False` in a suitable line within the
function `transition_directions_test` is not an alternative to passing the
argument `-Werror`, because the `assert False` will result in a traceback
where the `assert` statement appears, instead of a traceback that shows the
call stack at the point where the warning was issued.)


## Speeding up debugging using `pytest`

Also, since I had to be running `pytest` on the Python file `abstract_test.py`,
`pytest` would collect all test functions, and run them. The file
`abstract_test.py` happens to contain several slow test functions, so running
them all just to observe the results for the one function of interest is not
time-efficient.

What I did to speed up runs was to rename all `test_*` functions contained in
`abstract_test.py`, except for the one function of interest (namely
`transition_directions_test`), to identifiers outside the patterns collected
by `pytest`.

A simpler alternative, for use with larger test files, is to do the opposite:
rename only the function of interest to a different pattern, and then change
the line `python_functions = ` in the configuration file `pytest.ini`.


## References

- numpy/numpy#10142  (DEP: Pending deprecation warning for matrix)
- numpy/numpy#10973  (DOC: advise against use of matrix)
- scipy/scipy#8887  (MAINT: filter out np.matrix PendingDeprecationWarning's in numpy >=1.15)
- scipy/scipy#9734  (PendingDeprecationWarning for np.matrix with pytest)
- scikit-learn/scikit-learn#12327  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices)
- scikit-learn/scikit-learn#13076  ([MRG] Ignore PendingDepWarnings of matrix subclass with pytest)
- cvxpy/cvxpy#567  (NumPy matrix class is pending deprecation and issuing warnings)
- cvxpy/cvxpy#637  (RF: Use a 2D np array instead of matrix to represent scalars)
- cvxpy/cvxpy#638  (RF: Change np.matrix to np.array in several places)
- cvxpy/cvxpy#644  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra)
- https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning
johnyf added a commit to tulip-control/tulip-control that referenced this pull request Aug 12, 2021
@johnyf
BUG: use numpy.ndarray, instead of numpy.matrix
7da5534 
because matrices have been deprecated in `numpy`. Usage of `scipy.sparse` is
causing this issue when conversions to matrices are performed. I changed the:
- calls to the method `scipy.sparse.lil.lil_matrix.todense`
- to calls to the method `scipy.sparse.lil.lil_matrix.toarray`,

to avoid the conversions that raise `PendingDeprecationWarning`s. The warnings
are raised by the call to the method `lil_matrix.todense` because this call
involves the instantiation of the class `numpy.matrix`, which is deprecated.
In contrast, the method `lil_matrix.toarray` creates an instance of
`numpy.ndarray`.

(In fact, in the module `tulip.abstract.discretization`, wherever the method
`lil_matrix.todense` was called, the value that it returned was immediately
converted to a `numpy.ndarray`. So calling the method `lil_matrix.toarray` is
actually more efficient.)

The class `numpy.matrix` is deprecated and will probably be removed in
the future. This will happen after arrangements have been made for
`scipy.space`. (For these points and more information, read the references
listed at the end.)

Still, I do not think that continuing to use `scipy.sparse.lil_matrix` until
when `numpy` removes matrices is a safe approach.
Instead, using `numpy.ndarray` would be safer.
Moreover, I do think that there are other data structures that would fit
this use case better than sparse matrices.


## Diagnosis

I describe below the approach I (eventually) followed to debug this warning,
because finding the cause was difficult.

The issue is a `PendingDeprecationWarning` issued from `numpy`.
This warning is visible in `pytest` runs, but *not* when running the Python
test file directly. Moreover, `pytest` reports the warning, and from which
test function it originates. The warning itself reads (I have wrapped
the lines here):

```
===================================== warnings summary ======================================
abstract_test.py::transition_directions_test
  /.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69:
  PendingDeprecationWarning: the matrix subclass is not the recommended way to
  represent matrices or deal with linear algebra
  (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html).
  Please adjust your code to use regular ndarray.
    return matrix(data, dtype=dtype, copy=False)
```

So the line in `tulip` that causes the warning cannot be found from the
information contained in the warning.

The above `PendingDeprecationWarning` was introduced in `numpy` in commit:
    numpy/numpy@11e9d2a
This warning was then ignored in the module `scipy.sparse.__init__`, in `scipy` commit:
    scipy/scipy@a874bd5
It appears that this configuration of warnings by `scipy` interacts with
`pytest` complexly:

- when running with `pytest abstract_test.py`,
  the `PendingDeprecationWarning` is visible, but

- when running with `python -X dev -- abstract_test.py`,
  the `PendingDeprecationWarning` is invisible. This behavior is due to
  the call:

  ```python
  warnings.filterwarnings(
      'ignore',
      message='the matrix subclass is not the recommended way')
  ```

  within `scipy.sparse.__init__.py` (introduced in the `scipy` commit
  that was mentioned above). Read also:
  https://docs.python.org/3/library/exceptions.html#PendingDeprecationWarning
  https://www.python.org/dev/peps/pep-0565/

As a result, it is difficult to find the cause within `tulip` of this
`PendingDeprecationWarning`.


## Getting a traceback

The test function that triggered the `PendingDeprecationWarning` from `numpy`
was not failing, so there was no traceback that would indicate which line in
`tulip` caused the warning.

In addition, there was an earlier warning issued by `matplotlib`. So turning
warnings to errors with the argument `-Werror` would cause `pytest` to turn the
`matplotlib` warning into an error, and stop before the warning of interest:

```shell
pytest -Werror abstract_test.py
```

So first I removed the `matplotlib` warnings (temporarily), by commenting the
line `matplotlib.use('Agg')` in the file `abstract_test.py`. This made the
warning of interest to become the first warning. I then passed `-Werror` to
`pytest`, and this turned the `numpy` warning into an error, which produced
the traceback shown below:

(The cause of these `matplotlib` warnings (there are two) is in the package
`polytope`, and has been addressed there, in commit:
    tulip-control/polytope@c464818
These changes will become available to `tulip` with the next `polytope` release.
Until then, the CI tests of `tulip` will raise these `matplotlib` warnings.
These warnings could be explicitly ignored by using `with pytest.warns`.)

(The paths to `tulip` in the traceback below lead to the repository's `tulip`,
instead of a directory under Python's `site-packages`, because during this
phase of debugging I installed `tulip` with `pip install -e .`, to iterate
faster while debugging.)

```
../tulip/abstract/discretization.py:1666: in discretize_switched
    plot_mode_partitions(merged_abstr, show_ts, only_adjacent)
../tulip/abstract/discretization.py:1673: in plot_mode_partitions
    axs = swab.plot(show_ts, only_adjacent)
../tulip/abstract/discretization.py:187: in plot
    ax = ab.plot(show_ts, only_adjacent, color_seed)
../tulip/abstract/discretization.py:403: in plot
    ax = _plot_abstraction(self, show_ts, only_adjacent,
../tulip/abstract/discretization.py:446: in _plot_abstraction
    ax = ab.ppp.plot(
../tulip/abstract/prop2partition.py:600: in plot
    return plot_partition(
.../.virtualenvs/.../lib/python3.9/site-packages/polytope/plot.py:90: in plot_partition
    trans = nx.to_numpy_matrix(trans, nodelist=ppp2trans)
.../.virtualenvs/.../lib/python3.9/site-packages/networkx/convert_matrix.py:553: in to_numpy_matrix
    M = np.asmatrix(A, dtype=dtype)
.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69: in asmatrix
    return matrix(data, dtype=dtype, copy=False)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

subtype = <class 'numpy.matrix'>
data = array([[1., 0., 0., 0., 0., 0.],
       [0., 1., 1., 0., 1., 1.],
       [1., 0., 1., 1., 0., 1.],
       [1., 0., 0., 1., 0., 0.],
       [0., 0., 0., 0., 1., 1.],
       [1., 0., 0., 0., 0., 1.]])
dtype = None, copy = False

    def __new__(subtype, data, dtype=None, copy=True):
>       warnings.warn('the matrix subclass is not the recommended way to '
                      'represent matrices or deal with linear algebra (see '
                      'https://docs.scipy.org/doc/numpy/user/'
                      'numpy-for-matlab-users.html). '
                      'Please adjust your code to use regular ndarray.',
                      PendingDeprecationWarning, stacklevel=2)
E       PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.

.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:116: PendingDeprecationWarning
```

As the traceback shows, the issue is due to a call to the function
`networkx.to_numpy_matrix` within the function `polytope.plot.plot_partition`.
So avoiding this warning will be possible after the next release of the
package `polytope`.

(Note that inserting an `assert False` in a suitable line within the
function `transition_directions_test` is not an alternative to passing the
argument `-Werror`, because the `assert False` will result in a traceback
where the `assert` statement appears, instead of a traceback that shows the
call stack at the point where the warning was issued.)


## Speeding up debugging using `pytest`

Also, since I had to be running `pytest` on the Python file `abstract_test.py`,
`pytest` would collect all test functions, and run them. The file
`abstract_test.py` happens to contain several slow test functions, so running
them all just to observe the results for the one function of interest is not
time-efficient.

What I did to speed up runs was to rename all `test_*` functions contained in
`abstract_test.py`, except for the one function of interest (namely
`transition_directions_test`), to identifiers outside the patterns collected
by `pytest`.

A simpler alternative, for use with larger test files, is to do the opposite:
rename only the function of interest to a different pattern, and then change
the line `python_functions = ` in the configuration file `pytest.ini`.


## References

- numpy/numpy#10142  (DEP: Pending deprecation warning for matrix)
- numpy/numpy#10973  (DOC: advise against use of matrix)
- scipy/scipy#8887  (MAINT: filter out np.matrix PendingDeprecationWarning's in numpy >=1.15)
- scipy/scipy#9734  (PendingDeprecationWarning for np.matrix with pytest)
- scikit-learn/scikit-learn#12327  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices)
- scikit-learn/scikit-learn#13076  ([MRG] Ignore PendingDepWarnings of matrix subclass with pytest)
- cvxpy/cvxpy#567  (NumPy matrix class is pending deprecation and issuing warnings)
- cvxpy/cvxpy#637  (RF: Use a 2D np array instead of matrix to represent scalars)
- cvxpy/cvxpy#638  (RF: Change np.matrix to np.array in several places)
- cvxpy/cvxpy#644  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra)
- https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning
johnyf added a commit to tulip-control/tulip-control that referenced this pull request Aug 20, 2021
@johnyf
BUG: use numpy.ndarray, instead of numpy.matrix
4265d82 
because matrices have been deprecated in `numpy`. Usage of `scipy.sparse` is
causing this issue when conversions to matrices are performed. I changed the:
- calls to the method `scipy.sparse.lil.lil_matrix.todense`
- to calls to the method `scipy.sparse.lil.lil_matrix.toarray`,

to avoid the conversions that raise `PendingDeprecationWarning`s. The warnings
are raised by the call to the method `lil_matrix.todense` because this call
involves the instantiation of the class `numpy.matrix`, which is deprecated.
In contrast, the method `lil_matrix.toarray` creates an instance of
`numpy.ndarray`.

(In fact, in the module `tulip.abstract.discretization`, wherever the method
`lil_matrix.todense` was called, the value that it returned was immediately
converted to a `numpy.ndarray`. So calling the method `lil_matrix.toarray` is
actually more efficient.)

The class `numpy.matrix` is deprecated and will probably be removed in
the future. This will happen after arrangements have been made for
`scipy.space`. (For these points and more information, read the references
listed at the end.)

Still, I do not think that continuing to use `scipy.sparse.lil_matrix` until
when `numpy` removes matrices is a safe approach.
Instead, using `numpy.ndarray` would be safer.


## Diagnosis

I describe below the approach I (eventually) followed to debug this warning,
because finding the cause was difficult.

The issue is a `PendingDeprecationWarning` issued from `numpy`.
This warning is visible in `pytest` runs, but *not* when running the Python
test file directly. Moreover, `pytest` reports the warning, and from which
test function the warning originates. The warning itself reads (I have wrapped
the lines here):

```
===================================== warnings summary ======================================
abstract_test.py::transition_directions_test
  /.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69:
  PendingDeprecationWarning: the matrix subclass is not the recommended way to
  represent matrices or deal with linear algebra
  (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html).
  Please adjust your code to use regular ndarray.
    return matrix(data, dtype=dtype, copy=False)
```

So the line in `tulip` that causes the warning cannot be found from the
information contained in the warning.

The above `PendingDeprecationWarning` was introduced in `numpy` in commit:
    numpy/numpy@11e9d2a
This warning was then ignored in the module `scipy.sparse.__init__`, in `scipy` commit:
    scipy/scipy@a874bd5
It appears that this configuration of warnings by `scipy` interacts with
`pytest` complexly:

- when running with `pytest abstract_test.py`,
  the `PendingDeprecationWarning` is visible, but

- when running with `python -X dev -- abstract_test.py`,
  the `PendingDeprecationWarning` is invisible. This behavior is due to
  the call:

  ```python
  warnings.filterwarnings(
      'ignore',
      message='the matrix subclass is not the recommended way')
  ```

  within `scipy.sparse.__init__.py` (introduced in the `scipy` commit
  that was mentioned above). Read also:
  https://docs.python.org/3/library/exceptions.html#PendingDeprecationWarning
  https://www.python.org/dev/peps/pep-0565/

As a result, it is difficult to find the cause within `tulip` of this
`PendingDeprecationWarning`.


## Getting a traceback

The test function that triggered the `PendingDeprecationWarning` from `numpy`
was not failing, so there was no traceback that would indicate which line in
`tulip` caused the warning.

In addition, there was an earlier warning issued by `matplotlib`. So turning
warnings to errors with the argument `-Werror` would cause `pytest` to turn the
`matplotlib` warning into an error, and stop before the warning of interest:

```shell
pytest -Werror abstract_test.py
```

So first I removed the `matplotlib` warnings (temporarily), by commenting the
line `matplotlib.use('Agg')` in the file `abstract_test.py`. This made the
warning of interest to become the first warning. I then passed `-Werror` to
`pytest`, and this turned the `numpy` warning into an error, which produced
the traceback shown below:

(The cause of these `matplotlib` warnings (there are two) is in the package
`polytope`, and has been addressed there, in commit:
    tulip-control/polytope@c464818
These changes will become available to `tulip` with the next `polytope` release.
Until then, the CI tests of `tulip` will raise these `matplotlib` warnings.
These warnings could be explicitly ignored by using `with pytest.warns`.)

(The paths to `tulip` in the traceback below lead to the repository's `tulip`,
instead of a directory under Python's `site-packages`, because during this
phase of debugging I installed `tulip` with `pip install -e .`, to iterate
faster while debugging.)

```
../tulip/abstract/discretization.py:1666: in discretize_switched
    plot_mode_partitions(merged_abstr, show_ts, only_adjacent)
../tulip/abstract/discretization.py:1673: in plot_mode_partitions
    axs = swab.plot(show_ts, only_adjacent)
../tulip/abstract/discretization.py:187: in plot
    ax = ab.plot(show_ts, only_adjacent, color_seed)
../tulip/abstract/discretization.py:403: in plot
    ax = _plot_abstraction(self, show_ts, only_adjacent,
../tulip/abstract/discretization.py:446: in _plot_abstraction
    ax = ab.ppp.plot(
../tulip/abstract/prop2partition.py:600: in plot
    return plot_partition(
.../.virtualenvs/.../lib/python3.9/site-packages/polytope/plot.py:90: in plot_partition
    trans = nx.to_numpy_matrix(trans, nodelist=ppp2trans)
.../.virtualenvs/.../lib/python3.9/site-packages/networkx/convert_matrix.py:553: in to_numpy_matrix
    M = np.asmatrix(A, dtype=dtype)
.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:69: in asmatrix
    return matrix(data, dtype=dtype, copy=False)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

subtype = <class 'numpy.matrix'>
data = array([[1., 0., 0., 0., 0., 0.],
       [0., 1., 1., 0., 1., 1.],
       [1., 0., 1., 1., 0., 1.],
       [1., 0., 0., 1., 0., 0.],
       [0., 0., 0., 0., 1., 1.],
       [1., 0., 0., 0., 0., 1.]])
dtype = None, copy = False

    def __new__(subtype, data, dtype=None, copy=True):
>       warnings.warn('the matrix subclass is not the recommended way to '
                      'represent matrices or deal with linear algebra (see '
                      'https://docs.scipy.org/doc/numpy/user/'
                      'numpy-for-matlab-users.html). '
                      'Please adjust your code to use regular ndarray.',
                      PendingDeprecationWarning, stacklevel=2)
E       PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.

.../.virtualenvs/.../lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py:116: PendingDeprecationWarning
```

As the traceback shows, the issue is due to a call to the function
`networkx.to_numpy_matrix` within the function `polytope.plot.plot_partition`.
So avoiding this warning will be possible after the next release of the
package `polytope`.

(Note that inserting an `assert False` in a suitable line within the
function `transition_directions_test` is not an alternative to passing the
argument `-Werror`, because the `assert False` will result in a traceback
where the `assert` statement appears, instead of a traceback that shows the
call stack at the point where the warning was issued.)


## Speeding up debugging using `pytest`

Also, since I had to be running `pytest` on the Python file `abstract_test.py`,
`pytest` would collect all test functions, and run them. The file
`abstract_test.py` happens to contain several slow test functions, so running
them all just to observe the results for the one function of interest is not
time-efficient.

Running a single test function using `pytest` is possible by writing:

```shell
pytest abstract_test.py::name_of_function
```


## References

- numpy/numpy#10142  (DEP: Pending deprecation warning for matrix)
- numpy/numpy#10973  (DOC: advise against use of matrix)
- scipy/scipy#8887  (MAINT: filter out np.matrix PendingDeprecationWarning's in numpy >=1.15)
- scipy/scipy#9734  (PendingDeprecationWarning for np.matrix with pytest)
- scikit-learn/scikit-learn#12327  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices)
- scikit-learn/scikit-learn#13076  ([MRG] Ignore PendingDepWarnings of matrix subclass with pytest)
- cvxpy/cvxpy#567  (NumPy matrix class is pending deprecation and issuing warnings)
- cvxpy/cvxpy#637  (RF: Use a 2D np array instead of matrix to represent scalars)
- cvxpy/cvxpy#638  (RF: Change np.matrix to np.array in several places)
- cvxpy/cvxpy#644  (PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra)
- https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rth rth rth approved these changes

@qinhanmin2014 qinhanmin2014 qinhanmin2014 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@NicolasHug @rth @qinhanmin2014
0