8000 cumsum of mutable objects in an array gives wrong results · Issue #7402 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

cumsum of mutable objects in an array gives wrong results #7402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
siudej opened this issue Mar 10, 2016 · 5 comments · Fixed by #7407
Closed

cumsum of mutable objects in an array gives wrong results #7402

siudej opened this issue Mar 10, 2016 · 5 comments · Fixed by #7407

Comments

@siudej
Copy link
siudej commented Mar 10, 2016

While trying to replace Python long integers with gmpy2 mpz objects in an object array, I discovered a rather strange behavior of the in-place np.cumsum function (with out=). It seems that the objects are reused in some way. E.g.

import numpy as np
from gmpy2 import mpz

arr = np.empty(4, dtype=object)
arr[:] = [mpz(1) for _ in range(4)]
np.cumsum(arr, out=arr)
# np.cumsum(arr, out=arr)
print arr

prints [mpz(2) mpz(2) mpz(3) mpz(4)].

With the second cumsum uncommented, the values are 7, 7, 7, 11. There is no problem if long is used instead of mpz.

It seems that a similar issue can be reproduced using tuples and lists in an array.

import numpy as np

arr = np.empty(4, dtype=object)
arr[:] = [(1,) for _ in range(4)]
np.cumsum(arr, out=arr)
print arr  
arr[:] = [[1] for _ in range(4)]
np.cumsum(arr, out=arr)
print arr

As I would expect, the first print shows an array of tuples of growing length. However, the second print causes Segmentation fault: 11.
@jaimefrio
Copy link
Member

It works for me:

>>> np.cumsum(arr, out=arr)
array([[1], [1, 1], [1, 1, 1], [1, 1, 1, 1]], dtype=object)

What version of numpy and python and on what system are you using?

@siudej
Copy link
Author
8000
siudej commented Mar 10, 2016

I have tried on:
OS X 10.10, Python 2.7.10 (system one), numpy 1.8.0rc1
and
Same operating system, Anaconda 2.3, Python 2.7.11, numpy 1.10.2

Tomorrow I will try at work, on a Linux machine.

@seberg
Copy link
Member
seberg commented Mar 10, 2016

Can reproduce on python 2.7 at least with both 1.8 and master. It segfaults during the list concat. Best guess is that the refcount drops to zero and one of the list got cleaned up. Though I am not sure why that should happen. Might want to check python as well for some weirdness.

Valgrind trace:

==22710== Invalid read of size 8
==22710==    at 0x52F4F4: list_concat.lto_priv.99 (listobject.c:530)
==22710==    by 0x508C77: PyNumber_Add (abstract.c:1190)
==22710==    by 0x153FF935: PyUFunc_OO_O (loops.c.src:534)

@siudej siudej mentioned this issue Mar 10, 2016
Closed
@siudej
Copy link
Author
siudej commented Mar 10, 2016

I have checked on Linux Mint, Python 2.7.6, numpy 1.8.2. The same happens.

I have also checked with mpfr type (arbitrary precision floats) from gmpy2, and I get a list of NaNs from cumsum.

@seberg seberg mentioned this issue Mar 10, 2016
Merged
@seberg
Copy link
Member
seberg commented Mar 10, 2016

OK, found the bug, see gh-7407, simple decref before incref for the first element.

seberg added a commit to seberg/numpy that referenced this issue Mar 12, 2016
@seberg
BUG: Fix decref before incref for in-place accumulate
4443c4d 
closes numpygh-7402
charris added a commit that referenced this issue Mar 12, 2016
@charris
Merge pull request #7407 from seberg/gh-7402
f98b59e 
BUG: Fix decref before incref for in-place accumulate
charris pushed a commit to charris/numpy that referenced this issue Mar 12, 2016
@seberg @charris
BUG: Fix decref before incref for in-place accumulate
e428127 
closes numpygh-7402
@charris charris mentioned this issue Mar 12, 2016
Merged
jaimefrio pushed a commit to jaimefrio/numpy that referenced this issue Mar 22, 2016
@seberg @jaimefrio
BUG: Fix decref before incref for in-place accumulate
e2684d0 
closes numpygh-7402
@jaimefrio jaimefrio mentioned this issue Mar 26, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: Fix decref before incref for in-place accumulate seberg/numpy
3 participants
@seberg @jaimefrio @siudej
0