Fancy slicing inconsistent behaviour with lists of lists and arrays of arrays #7909

henpemaz · 2016-08-05T16:44:24Z

Hello !

I am working on some triangular mesh processing, and I've stumbled on some weird behaviour for some of our smaller geometry tests.

Apparently, lists of lists and arrays of arrays behave very differently for fancy queries up to a length of 32.

Here is some code to reproduce the issue:

import numpy as np
import random as rd

# The idea is to use a list of [v1, v2, v3] faces to query the vertices in groups of 3, for easy use in geometrical ops
# The result of the fancy slicing must be an array of len-3 arrays of vertices (also len-3 arrays)
# In other words, a big array containing 3-by-3 arrays of coordinates for each face
vertices = np.zeros((1000,3))  # Apparently, the size of the array being queried does not matter
face = lambda : [rd.randint(0, 999), rd.randint(0, 999), rd.randint(0, 999)]

i = 2  # If trying to slice with a 1-by-3 list, this returns a single 3-by-3 array, even given the higher dimension of the list (expected result would be a 1-by-3-by-3 array)
       # Yet this is an "understandable" behaviour, even if wrong and misleading

while True:
    faces_list = [face() for f in range(i)]
    assert vertices[np.array(faces_list)].shape == (i, 3, 3) # This works as expected and will not throw any exception

    try:
        my_vertices = vertices[faces_list]  # This behaves as some sort of multi-dimensional slicing instead, up to a certain limit of dimensions, then it starts behaving as expected
        assert my_vertices.shape == (i, 3, 3)  # Does not fail if the line above worked as intended
        break
    except:  # The block above throws "IndexError: too many indices for array"
        print("numpy will not handle a list with {} lists for fancy slicing :^/".format(i))
        i += 1

print("numpy can handle a list with {} or more lists :)".format(i))
exit(0)

I don't use numpy arrays for my faces for reasons that are not relevant here, and I hadn't run into this issue until I've tested my solution on very tiny meshes.
I can easily contour this problem by making a numpy copy of the faces list (since it's so small in this case), so I don't actually need a fix, but I do think it's a bug.

I'm running on Ubuntu 14.4, Python 3.4, and np.version.version == '1.8.2'

Good luck, and thanks for your hard work

seberg · 2016-08-05T16:53:52Z

Lists of lists are interpreted as high dimensional indices. It is something that comes up every once in a while, though nice tracking it down even to the magic 32 number ;). Got even an open PR to deprecate non-tuple multi dimensional indices. The most annoying part of which is that it is even used in NumPy a gazillion times (and weeding out all the warnings is hell). Also I guess since nobody tried, we don't know how much people use the list thing. It is quite common, since sometimes you want to change the indexing tuples, and then using lists instead of tuples happens quickly.

For reference see gh-4434 please feel free to keep discussing, but will close this anyway.

seberg · 2016-08-05T16:54:48Z

If this annoys you, the solution is to do arr[list_of_list,]. Adding the extra , makes things clear for numpy.

henpemaz · 2016-08-05T17:21:27Z

"It's such a nice bug, it's almost like a feature..."

Well, I don't have any problems with that, and tracking it down wasn't hard, but it could be made clearer in the Advanced Slicing documentation that there is a difference if you use ndarrays or lists...

Thanks for the quick answer

seberg · 2016-08-05T17:26:00Z

Heh feature.... It really annoys me, but it is a feature explicitly put in at some point ;). If you have some time or proposals, documentation improvements are always welcome!

seberg closed this as completed Aug 5, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fancy slicing inconsistent behaviour with lists of lists and arrays of arrays #7909

Fancy slicing inconsistent behaviour with lists of lists and arrays of arrays #7909

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fancy slicing inconsistent behaviour with lists of lists and arrays of arrays #7909

Fancy slicing inconsistent behaviour with lists of lists and arrays of arrays #7909

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!