[BUG] Label propagation sometimes produces label_distributions that contain Nan. #9292

alpapado · 2017-07-07T18:24:33Z

Description

Invalid value encountered in true_divide through when calling fit on LabelSpreading.

After convergence, the label distribution for some samples is all zero and so the variable normalizer in label_propagation.py:291 contains some zero values causing the division self.label_disributions_ /= normalizer to produce NaN.

Maybe there is a connection to #8008? In other datasets, increasing the n_neighbors parameter to a larger than the default value, caused the issue not to appear.

Steps/Code to Reproduce

from sklearn.datasets import fetch_mldata
from sklearn.semi_supervised import label_propagation
import numpy

numpy.seterr(all='raise')

mnist = fetch_mldata('MNIST original', data_home="./tmp")

X = mnist.data[1:10000]
y = mnist.target[1:10000]

# Use only 300 labeled examples
y[300:] = -1

lp_model = label_propagation.LabelSpreading(kernel='knn', n_neighbors=7, n_jobs=-1)
lp_model.fit(X,y)

Expected Results

No error is thrown.

Actual Results

  File "reproduce.py", line 16, in <module>
    lp_model.fit(X,y)
  File "...anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/semi_supervised/label_propagation.py", line 291, in fit
    self.label_distributions_ /= normalizer
FloatingPointError: invalid value encountered in true_divide

Versions

[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
NumPy 1.13.0
SciPy 0.19.0
Scikit-Learn 0.19.dev0

The text was updated successfully, but these errors were encountered:

jnothman · 2017-07-08T10:51:58Z

Is this on the current master? we've changed LabelSpreading a lot. Ping @musically-ut.

…

On 8 July 2017 at 04:24, Alexandros Papadopoulos ***@***.***> wrote: Description Invalid value encountered in true_divide through when calling fit on LabelSpreading. The variable normalizer in label_propagation.py:291 contains some zero values and because of that, the division self.label_disributions_ /= normalizer produces NaN. Maybe there is a connection to #8008 <#8008>? In other datasets, increasing the n_neighbors parameter to a larger than the default value, caused the issue not to appear. Steps/Code to Reproduce from sklearn.datasets import fetch_mldatafrom sklearn.semi_supervised import label_propagationimport numpy numpy.seterr(all='raise') mnist = fetch_mldata('MNIST original', data_home="./tmp") X = mnist.data[1:10000] y = mnist.target[1:10000] # Use only 300 labeled examples y[300:] = -1 lp_model = label_propagation.LabelSpreading(kernel='knn', n_neighbors=7, n_jobs=-1) lp_model.fit(X,y) Expected Results No error is thrown. Actual Results File "reproduce.py", line 16, in <module> lp_model.fit(X,y) File "...anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/semi_supervised/label_propagation.py", line 291, in fit self.label_distributions_ /= normalizer FloatingPointError: invalid value encountered in true_divide Versions [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] NumPy 1.13.0 SciPy 0.19.0 Scikit-Learn 0.19.dev0 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9292>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz69YGncd_yQGsAxAiaaQw6qMW1qhZks5sLnfjgaJpZM4ORRzj> .

musically-ut · 2017-07-08T10:55:42Z

The knn kernel is a bit circumspect because of the directed edges it produces and a fix is in the works.

Can this be reproduced with the rbf kernel as well?

alpapado · 2017-07-08T11:46:34Z

Yes I believe it is on the current master. That is, it contains the latest pull request regarding the label_propagation module.

In the above code, changing the kernel to rbf produces a different, although similar, error:

Traceback (most recent call last):
  File "reproduce.py", line 21, in <module>
    lp_model.fit(X,y)
  File "/share/mug/gentoo/anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/semi_supervised/label_propagation.py", line 234, in fit
    graph_matrix = self._build_graph()
  File "/share/mug/gentoo/anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/semi_supervised/label_propagation.py", line 511, in _build_graph
    affinity_matrix = self._get_kernel(self.X_)
  File "/share/mug/gentoo/anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/semi_supervised/label_propagation.py", line 131, in _get_kernel
    return rbf_kernel(X, X, gamma=self.gamma)
  File "/share/mug/gentoo/anaconda3/envs/ssl-py3/lib/python3.6/site-packages/sklearn/metrics/pairwise.py", line 837, in rbf_kernel
    np.exp(K, K)    # exponentiate K in-place
FloatingPointError: underflow encountered in exp

musically-ut · 2017-07-08T11:56:14Z

Changing gamma should be able to circumvent the underflow in case of rbf kernel. We do need more tests for numerical stability in both cases.

jnothman · 2017-07-08T23:00:36Z

How might we avoid that underflow while calculating the kernel?

musically-ut · 2017-07-10T09:27:17Z

I think detecting underflows in the kernel is a separate issue. Is there an example of any method in scikit-learn which detects under/over flows?

In this particular case, I think underflows can be safely set to zero; it would just mean that certain nodes are not connected to each other. We can use numpy.seterr to "ignore" underflows before the call to the kernel and reset default behavior immediately afterwards.

Sounds reasonable?

jnothman · 2017-07-10T11:47:26Z

There appear to be at least a couple of cases where we explicitly ignore underflow

jnothman · 2017-07-10T11:48:27Z

Yes, your solution to underflow seems reasonable. Not the same as this issue, though

musically-ut · 2017-07-10T11:59:56Z

Hmm, I've been thinking about edge cases here.

If we end up with a connected component which does not have any labeled nodes within it, then the normalization will produce NaNs since the row of label_distribution_ will contain zeros irrespective of how many iterations we run the algorithm for (there are no nodes which can transduce any labels).

I think it is reasonable to give the output 1/n probability for all classes, but I am not sure how predict should break ties. Are there any examples of such tie-breaking?

alpapado · 2017-07-10T12:15:24Z

If we end up with a connected component which does not have any labeled nodes within it, then the normalization will produce NaNs since the row of label_distribution_ will contain zeros irrespective of how many iterations we run the algorithm for (there are no nodes which can transduce any labels).

I believe this is exactly the issue I am facing. It agrees with the observation that increasing the n_neighbors parameter to large enough values, prevents the error from occuring, which makes sense since increasing n_neighbors decreases the chances of ending up with a node that is not connected to any labeled nodes.

irenelizeth · 2018-02-22T03:59:16Z

I have replicated this issue when instantiating the LabelSpreading model with the default parameter values, i.e., LabelSpreading(). When I switch to instantiate it with LabelSpreading(gamma=0.25, max_iter=5) then the error is not thrown. Even when using gamma=0, max_iter=1 to instantiate LabelSpreading works fine just not defining the values for those parameters generates the issue:
label_propagation.py:293: RuntimeWarning: invalid value encountered in divide
self.label_distributions_ /= normalizer

musically-ut · 2018-10-29T11:09:26Z

Since some other people have also faced this issue (personal communication), I think it would be best to take the following steps:

Produce a warning if np.any(normalizer == 0) for either LabelPropagation or LabelSpreading which tells the user to either increase n_neighbors (for knn kernel) or decrease gamma (for rbf kernel).
Replace label_distribution_[:, idx] if normalizer[idx] == 0 with 1/n.
During prediction, select the first class arbitrarily (default behavior of np.argmax).

How does that sound?

jnothman · 2018-10-30T07:19:26Z

Is 1/n more reasonable than the empirical distribution of known labels?

musically-ut · 2018-11-06T23:26:43Z

1/n_empirical does sound more reasonable, and gives us a reasonable way of breaking the ties in .predict, the most likely class will be selected always.

Any comments about whether to show a warning as above or whether to throw an error (making the point moot) in case np.any(normalizer == 0)?

jnothman · 2018-11-07T04:26:52Z

Well I don't think a failure to infer labels on some points should break the whole point. I would raise a UserWarning and assign arbitrary labels.

ni-apurata · 2019-11-25T17:22:10Z

I'm also having serious problems with this bug. Has there been any progress towards fixing it or finding workarounds?

ra312 · 2020-07-29T07:28:41Z

I am just wonderinf if this issue has been fixed ? Any updates? Thanks!

…t contain Nan.(scikit-learn#9292)

jnothman added the Bug label Jul 8, 2017

musically-ut mentioned this issue Jul 10, 2017

[BUG] Underflows in LabelPropagation and LabelSpreading should be ignored. #9313

Closed

ThuWangzw added a commit to ThuWangzw/scikit-learn that referenced this issue Jan 25, 2021

Bug fix: Label propagation sometimes produces label_distributions tha…

a848ed7

…t contain Nan.(scikit-learn#9292)

ThuWangzw mentioned this issue Jan 25, 2021

FIX nan bug in BaseLabelPropagation #19271

Merged

rth closed this as completed in #19271 Feb 1, 2021

jeremiedbb mentioned this issue Mar 31, 2022

TST replace pytest.warns(None) in test_label_propagation.py #22998

Closed

CatherineTaelman mentioned this issue Feb 8, 2023

Ice-Water Classification from Sentinel-1 SAR imagery CryosphereVirtualLab/public-notebooks#33

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Label propagation sometimes produces label_distributions that contain Nan. #9292

[BUG] Label propagation sometimes produces label_distributions that contain Nan. #9292

[BUG] Label propagation sometimes produces label_distributions that contain Nan. #9292

[BUG] Label propagation sometimes produces label_distributions that contain Nan. #9292

Comments

Description

Steps/Code to Reproduce

Expected Results

Actual Results

Versions