8000 PCA singular value non-deterministic test failure on Appveyor · Issue #7893 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

PCA singular value non-deterministic test failure on Appveyor #7893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
amueller opened this issue Nov 16, 2016 · 9 comments · Fixed by #9162
Closed

PCA singular value non-deterministic test failure on Appveyor #7893

amueller opened this issue Nov 16, 2016 · 9 comments · Fixed by #9162
Labels
Bug Build / CI
Milestone
0.19

Comments

@amueller
Copy link
Member
amueller commented Nov 16, 2016 •
edited by jnothman
Loading

See here:
https://ci.appveyor.com/project/sklearn-ci/scikit-learn/build/1.0.10151/job/k042tl5h0hp91iv9

======================================================================
FAIL: sklearn.decomposition.tests.test_pca.test_singular_values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python35\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python35\lib\site-packages\sklearn\decomposition\tests\test_pca.py", line 266, in test_singular_values
    np.linalg.norm(X_apca, "fro")**2.0, 12)
  File "C:\Python35\lib\site-packages\numpy\testing\utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "C:\Python35\lib\site-packages\numpy\testing\utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 12 decimals
(mismatch 100.0%)
 x: array(613.0583877392909)
 y: array(613.0583877392893)
sklearn.ensemble.tests.test_bagging.test_base_estimator: 5.2514s
@amueller amueller added this to the 0.19 milestone Nov 16, 2016
@amueller amueller mentioned this issue Nov 16, 2016
Merged
@lesteve
Copy link
Member
lesteve commented Nov 17, 2016

I may be missing something but I think the build you are linking to is not a build from master but from the PR #7872 ...

This is the latest build from master I can find on AppVeyor and it is all green:
https://ci.appveyor.com/project/sklearn-ci/scikit-learn/build/1.0.10236

@amueller
Copy link
Member Author

The build is from a PR that didn't change anything in PCA though. I guess it's a non-deterministic failure?

@amueller
Copy link
Member Author

which is weird since it's arpack and we set a random state? I'm not entirely sure what the status of our initialization for arpack svds is though, I know there were some recent changes...

@dalmia
Copy link
Contributor
dalmia commented Nov 29, 2016

@amueller I want to work on the issue. Could you please guide me as to how can I get started with working on this?

@lesteve
Copy link
Member
lesteve commented Nov 29, 2016

I just noticed it happened more recently on master:
https://ci.appveyor.com/project/sklearn-ci/scikit-learn/build/1.0.10302/job/p5gdb45li72pe6bi

@dalmia this happens on Windows only and not deterministically. Maybe not the best issue if you do not have a Windows machine.

@dalmia
Copy link
Contributor
dalmia commented Nov 29, 2016

@amueller Yes, I don't. Thanks for letting me know.

@lesteve lesteve changed the title PCA singluar value test failure on master PCA singular value non-deterministic test failure on Appveyor Nov 29, 2016
@qinhanmin2014
Copy link
Member

It happened again in https://ci.appveyor.com/project/sklearn-ci/scikit-learn/build/1.0.14629/job/t8udw2qky46u5ewm (#9108). CI become green after an empty commit. The wrong result in three cases are exactly the same(wrong result:613.0583877392909, right result:613.0583877392893). In order to prevent such confusion, do you agree to change the decimal restriction in assert_array_almost_equal from 12 to 6(default value) ? It seems reasonable to me. @amueller @lesteve

@qinhanmin2014 qinhanmin2014 mentioned this issue Jun 19, 2017
Merged
@jnothman
Copy link
Member
jnothman commented Jun 19, 2017 via email

@qinhanmin2014
Copy link
Member

@jnothman
Thanks. I also feel strange. I read the source code and don't find any mistake currently. Since the error only happen for python3.5 in 32-bit Windows, maybe it's a bug for python3.5(new python) in 32-bit Windows(old Windows). With your support, I'll make a pull request today to close the issue.

@qinhanmin2014 qinhanmin2014 mentioned this issue Jun 19, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Build / CI
Projects
None yet
Milestone
0.19
Development

Successfully merging a pull request may close this issue.

[MRG+1] reduce decimal restriction in test_singular_values to avoid non-deterministic test failure
5 participants
@jnothman @amueller @lesteve @qinhanmin2014 @dalmia
0