8000 TST: do one of the test runs on TravisCI with latest numpy master. by rgommers · Pull Request #4729 · scipy/scipy · GitHub
[go: up one dir, main page]
Skip to content

TST: do one of the test runs on TravisCI with latest numpy master. #4729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

TST: do one of the test runs on TravisCI with latest numpy master. #4729

wants to merge 1 commit into from

Conversation

rgommers
Copy link
Member

This should prevent build issues like gh-4719, as well as catch test errors/failures that are regularly introduced by PRs.

Note: this will fail on a number of test errors. Some of those are fixed by an open PR, some are to be fixed (maybe in this PR). Want to see already if this runs though, and if the errors and failures on TravisCI are the same as those I see locally.

Closes gh-4327

@rgommers rgommers added the maintenance Items related to regular maintenance tasks label Apr 13, 2015
@argriffing
Copy link
Contributor

This would close #4327.

@argriffing
Copy link
Contributor

The PR has been stalled in TravisCI for 2+ hours, and only the pyflakes config has become active.

@rgommers
Copy link
Member Author

Finished now. It's green, which is yet another problem (I'd like red on a segfault....):

$ python $OPTIMIZE runtests.py -g -m $TESTMODE $COVERAGE
Building, see build.log...
    ... build in progress
    ... 
Build OK
Running unit tests for scipy
NumPy version 1.10.0.dev0+0752872
NumPy relaxed strides checking option: True
NumPy is installed in /home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/numpy
SciPy version 0.16.0.dev0+7e713e3
...
....................................................................................
....... ** On entry to ZGEMM parameter number 10 had an illegal value
The command "python $OPTIMIZE runtests.py -g -m $TESTMODE $COVERAGE" exited with 0.
after_success
0.01s
$ if [ "${TESTMODE}" == "full" ]; then pushd build/testenv/lib/python*/site-packages/ 
&& cp ../../../../test/.coverage . && coveralls && popd; fi
Done. Your build exited with 0.

@ev-br
Copy link
Member
ev-br commented Apr 14, 2015

FWIW, on my 64bit ubuntu 12.04 numpy 1.9.1 does not produce any error.
With numpy-dev:

$ python runtests.py -t scipy/linalg/tests/test_decomp_update.py:TestQRupdate_D  -v
Building, see build.log...
Build OK
Running unit tests for scipy
NumPy version 1.10.0.dev0+0752872
NumPy relaxed strides checking option: True
NumPy is installed in /home/br/virtualenvs/scipy-dev/local/lib/python2.7/site-packages/numpy
SciPy version 0.16.0.dev0+4f65e8e
SciPy is installed in /home/br/installs/scipy/build/testenv/lib/python2.7/site-packages/scipy
Python version 2.7.3 (default, Sep 26 2013, 20:03:06) [GCC 4.6.3]
nose version 1.3.4
test_decomp_update.TestQRupdate_D.test_check_finite ... ok
test_decomp_update.TestQRupdate_D.test_economic_check_finite ... ok
test_decomp_update.TestQRupdate_D.test_economic_rank_1 ... ok
test_decomp_update.TestQRupdate_D.test_economic_rank_p ... ok
test_decomp_update.TestQRupdate_D.test_empty_inputs ... ok
test_decomp_update.TestQRupdate_D.test_fat_rank_1 ... ok
test_decomp_update.TestQRupdate_D.test_fat_rank_p ...  ** On entry to ZGEMM parameter number 10 had an illegal value

This was referenced Apr 14, 2015
Closed
Closed
@ev-br
Copy link
Member
ev-br commented May 2, 2015

Relaxed strides checking runs on Travisbot. This would fail (at least) on qr_update and IDL.
Merge? Y/n?

@rgommers
Copy link
Member Author
rgommers commented May 3, 2015

Let's hold off on this one for a bit I'd say. Merge when it's won't immediately cause failures, which should be soon I think (or we'll not be branching...)

@rgommers rgommers force-pushed the travisci-numpydev branch 2 times, most recently from a35e3da to 2f34dcc Compare May 10, 2015 15:34
TST: do one of the test runs on TravisCI with latest numpy master.
2f34dcc 
This should prevent build issues like scipygh-4719, as well as catch
test errors/failures that are regularly introduced by PRs.
@rgommers
Copy link
Member Author

The tests failed but that didn't get detected. One new failure with numpy master:

======================================================================
FAIL: test_data.test_boost(<Data for arccosh (complex): acosh_data_ipp-acosh_data>,)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/travis/build/scipy/scipy/build/testenv/lib/python3.3/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory
    test.check(dtype=dtype)
  File "/home/travis/build/scipy/scipy/build/testenv/lib/python3.3/site-packages/scipy/special/_testutils.py", line 292, in check
    assert_(False, "\n".join(msg))
  File "/home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 
Max |adiff|: 1.77636e-15
Max |rdiff|: 1.09352e-13
Bad results (1 out of 261) for the following points (in output 0):
              (1.0000014305114746+0j) =>     (0.0016914556651294794+0j) !=        0.0016914556651292944  (rdiff         1.0935249113484058e-13)

rtol in the complex test is now 5e-14, so should be upped a bit. Unless someone feels like this needs investigating of course.

@rgommers
Copy link
Member Author

The problem of having the TravisCI run fail because of an issue in numpy master can be worked around by allow_failure (see numpy/numpy#6462 (comment))

@argriffing
Copy link
Contributor

Unless someone feels like this needs investigating of course.

I investigated this a bit. My guess is that when numpy is configured or built on the TravisCI platform it somehow fails to detect a C library implementation of acosh, and so it is falling back to a naive hardcoded implementation.

The failing test is from the first line of https://github.com/scipy/scipy/blob/master/scipy/special/tests/data/boost/acosh_data_ipp/acosh_data.txt. Here's what I see when I manually run this test on my system where numpy seems to have been able to find a clever acosh:

>>> import numpy as np
>>> from numpy import log, sqrt
>>> def fallback(x): return 2*log(sqrt((x+1)/2) + sqrt((x-1)/2))
... 
>>> x = 1.000001430511474609375
>>> y = 0.001691455665129294448190238354291760044433
>>> (fallback(x) - y) / y
1.0935249113484058e-13
>>> (np.arccosh(x) - y) / y
0.0

The fallback function in that code is a re-implementation of the algorithm that numpy uses when HAVE_ACOSH is not defined in https://github.com/numpy/numpy/blob/master/numpy/core/src/npymath/npy_math.c.src. Notice that its relative error of 1.0935249113484058e-13 is the same as the rdiff reported in the failing test (for example in https://travis-ci.org/scipy/scipy/jobs/92603062):

Bad results (1 out of 261) for the following points (in output 0):
(1.0000014305114746+0j) => (0.0016914556651294794+0j) != 0.0016914556651292944
(rdiff 1.0935249113484058e-13)

Python's fallback implementations are here: https://github.com/python/cpython/blob/master/Modules/_math.c.

One workaround could be to increase the scipy test tolerance for arccosh. A better workaround could be to improve numpy's fallback implementation. An even better solution would be to have numpy's build process somehow detect the right library function (assuming it's present on the platform).

@argriffing
Copy link
Contributor

Numpy undefines some complex arc trig functions when the glibc version is not at least 2.16 to avoid bugs in old standard C math libraries.
https://github.com/numpy/numpy/blob/master/numpy/core/src/private/npy_config.h#L78
TravisCI uses Ubuntu 12.04 which according to the internet has glibc version 2.15. I think this is somehow the reason that the numpy fallback arccosh implementation is being used.

This was referenced Nov 23, 2015
Closed
Merged
@argriffing
Copy link
Contributor

rtol in the complex test is now 5e-14, so should be upped a bit.

upped in #5527

@ev-br
Copy link
Member
ev-br commented Dec 30, 2015

Seems to be superseded by #5609.

@argriffing
Copy link
Contributor

closing; #5609

@argriffing argriffing closed this Dec 31, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
maintenance Items related to regular maintenance tasks
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

On one of the TravisCI configs, treat numpy deprecation warnings as errors or failures.
3 participants
@rgommers @argriffing @ev-br
0