8000 Raise ValueError instead of RuntimeWarning for LinearModelLoss by akaashp2000 · Pull Request #27332 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Raise ValueError instead of RuntimeWarning for LinearModelLoss #27332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Raise ValueError instead of RuntimeWarning for LinearModelLoss #27332

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5601fb5
Raise ValueError instead of RuntimeWarning
akaashp2000 Sep 10, 2023
09e27d2
scale X from load_breast_cancer to avoid convergence error
akaashp2000 Sep 10, 2023
428c736
Remove ignore ConvergenceWarning from tests based on diabetes data
akaashp2000 Sep 10, 2023
0721f6b
Linting fix
akaashp2000 Sep 10, 2023
9585d68
Simplify scaling of X dataframe
akaashp2000 Sep 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions sklearn/ensemble/tests/test_stacking.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
StackingClassifier,
StackingRegressor,
)
from sklearn.exceptions import ConvergenceWarning, NotFittedError
from sklearn.exceptions import NotFittedError
from sklearn.linear_model import (
LinearRegression,
LogisticRegression,
Expand All @@ -42,7 +42,6 @@
from sklearn.utils._testing import (
assert_allclose,
assert_allclose_dense_sparse,
ignore_warnings,
)
from sklearn.utils.fixes import COO_CONTAINERS, CSC_CONTAINERS, CSR_CONTAINERS

Expand All @@ -54,6 +53,8 @@
n_classes=3, random_state=42
)
X_binary, y_binary = make_classification(n_classes=2, random_state=42)
X_breast_cancer, y_breast_cancer = load_breast_cancer(return_X_y=True)
X_breast_cancer = scale(X_breast_cancer)


@pytest.mark.parametrize(
Expand Down Expand Up @@ -435,7 +436,8 @@ def test_stacking_classifier_stratify_default():
final_estimator=LogisticRegression(),
cv=KFold(shuffle=True, random_state=42),
),
*load_breast_cancer(return_X_y=True),
X_breast_cancer,
y_breast_cancer,
),
(
StackingRegressor(
Expand Down Expand Up @@ -464,18 +466,15 @@ def test_stacking_with_sample_weight(stacker, X, y):
X, y, total_sample_weight, random_state=42
)

with ignore_warnings(category=ConvergenceWarning):
stacker.fit(X_train, y_train)
stacker.fit(X_train, y_train)
y_pred_no_weight = stacker.predict(X_test)

with ignore_warnings(category=ConvergenceWarning):
stacker.fit(X_train, y_train, sample_weight=np.ones(y_train.shape))
stacker.fit(X_train, y_train, sample_weight=np.ones(y_train.shape))
y_pred_unit_weight = stacker.predict(X_test)

assert_allclose(y_pred_no_weight, y_pred_unit_weight)

with ignore_warnings(category=ConvergenceWarning):
stacker.fit(X_train, y_train, sample_weight=sample_weight_train)
stacker.fit(X_train, y_train, sample_weight=sample_weight_train)
y_pred_biased = stacker.predict(X_test)

assert np.abs(y_pred_no_weight - y_pred_biased).sum() > 0
Expand All @@ -490,7 +489,6 @@ def test_stacking_classifier_sample_weight_fit_param():
stacker.fit(X_iris, y_iris, sample_weight=np.ones(X_iris.shape[0]))


@pytest.mark.filterwarnings("ignore::sklearn.exceptions.ConvergenceWarning")
@pytest.mark.parametrize(
"stacker, X, y",
[
Expand All @@ -502,7 +500,8 @@ def test_stacking_classifier_sample_weight_fit_param():
],
final_estimator=LogisticRegression(),
),
*load_breast_cancer(return_X_y=True),
X_breast_cancer,
y_breast_cancer,
),
(
StackingRegressor(
Expand Down
20 changes: 18 additions & 2 deletions sklearn/linear_model/_linear_loss.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,13 +291,29 @@

if not self.base_loss.is_multiclass:
grad = np.empty_like(coef, dtype=weights.dtype)
grad[:n_features] = X.T @ grad_pointwise + l2_reg_strength * weights
with np.errstate(all="raise"):
try:
grad[:n_features] = X.T @ grad_pointwise + l2_reg_strength * weights
except FloatingPointError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except FloatingPointError:
except FloatingPointError as exc:

Copy link
@akaashpatelmns akaashpatelmns Nov 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I use exc as you have in this suggestion, testing on the dataset from I get the following:

---------------------------------------------------------------------------
FloatingPointError                        Traceback (most recent call last)
File [~/Downloads/scikit-learn/sklearn/linear_model/_linear_loss.py:296](https://file+.vscode-resource.vscode-cdn.net/Users/d0055315_akaash_patel/Downloads/scikit-learn/scratchpad~/~/Downloads/scikit-learn/sklearn/linear_model/_linear_loss.py:296), in LinearModelLoss.loss_gradient(self, coef, X, y, sample_weight, l2_reg_strength, n_threads, raw_prediction)
    [295] try:
--> [296]     grad[:n_features] = X.T @ grad_pointwise + l2_reg_strength * weights
    [297]     print("min entry", np.min(grad_pointwise))

FloatingPointError: invalid value encountered in matmul

The above exception was the direct cause of the following exception:

so it still includes the original exception that I am "wrapping" with a ValueError and message. Do you reckon it's better this way?

raise ValueError(

Check warning on line 298 in sklearn/linear_model/_linear_loss.py

View check run for this annotation

Codecov / codecov/patch

sklearn/linear_model/_linear_loss.py#L297-L298 

Added lines #L297 - L298 were not covered by tests
"Overflow detected. Try scaling the target variable or"
" features, or using a different solver"
) from None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
) from None
) from exc

if self.fit_intercept:
grad[-1] = grad_pointwise.sum()
else:
grad = np.empty((n_classes, n_dof), dtype=weights.dtype, order="F")
# grad_pointwise.shape = (n_samples, n_classes)
grad[:, :n_features] = grad_pointwise.T @ X + l2_reg_strength * weights
with np.errstate(all="raise"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is an overflow, I think that using over="raise" would be better.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried using over="raise", however this didn't raise an Exception and the RuntimeWarning appeared instead - when I tested with the dataset I encountered this issue with in the first place #27016. During one of the iterations, there was an infinity in one the gradients which is where I wanted to raise the Exception.

try:
grad[:, :n_features] = (
grad_pointwise.T @ X + l2_reg_strength * weights
)
except FloatingPointError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except FloatingPointError:
except FloatingPointError as exc:

raise ValueE 67E6 rror(

Check warning on line 313 in sklearn/linear_model/_linear_loss.py

View check run for this annotation

Codecov / codecov/patch

sklearn/linear_model/_linear_loss.py#L312-L313 

Added lines #L312 - L313 were not covered by tests
"Overflow detected. Try scaling the target variable or"
" features, or using a different solver"
) from None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
) from None
) from exc

if self.fit_intercept:
grad[:, -1] = grad_pointwise.sum(axis=0)
if coef.ndim == 1:
Expand Down
Loading
0