8000 PyTables: any limitation on the number of columns? · Issue #7653 · pandas-dev/pandas · GitHub
[go: up one dir, main page]

Skip to content
PyTables: any limitation on the number of columns? #7653
Closed
@bluefir

Description

@bluefir

I got the following error trying to do

store.append(output_name, data_frame)

for a DataFrame with 2 rows and 3202 columns:

  File "C:\Python27\lib\site-packages\pandas\io\pytables.py", line 914, in append
    **kwargs)
  File "C:\Python27\lib\site-packages\pandas\io\pytables.py", line 1273, in _write_to_group
    s.write(obj=value, append=append, complib=complib, **kwargs)
  File "C:\Python27\lib\site-packages\pandas\io\pytables.py", line 3589, in write
    self.set_attrs()
  File "C:\Python27\lib\site-packages\pandas\io\pytables.py", line 2887, in set_attrs
    self.attrs.non_index_axes = self.non_index_axes
  File "C:\Python27\lib\site-packages\tables\attributeset.py", line 455, in __setattr__
    self._g__setattr(name, value)
  File "C:\Python27\lib\site-packages\tables\attributeset.py", line 397, in _g__setattr
    self._g_setattr(self._v_node, name, stvalue)
  File "hdf5extension.pyx", line 704, in tables.hdf5extension.AttributeSet._g_setattr (tables\hdf5extension.c:6832)
tables.exceptions.HDF5ExtError: HDF5 error back trace

  File "..\..\hdf5-1.8.11\src\H5A.c", line 254, in H5Acreate2
    unable to create attribute
  File "..\..\hdf5-1.8.11\src\H5A.c", line 503, in H5A_create
    unable to create attribute in object header
  File "..\..\hdf5-1.8.11\src\H5Oattribute.c", line 347, in H5O_attr_create
    unable to create new attribute in header
  File "..\..\hdf5-1.8.11\src\H5Omessage.c", line 224, in H5O_msg_append_real
    unable to create new message
  File "..\..\hdf5-1.8.11\src\H5Omessage.c", line 1945, in H5O_msg_alloc
    unable to allocate space for message
  File "..\..\hdf5-1.8.11\src\H5Oalloc.c", line 1142, in H5O_alloc
    object header message is too large

End of HDF5 error back trace

Can't set attribute 'non_index_axes' in node:
 /alpha_regression_results/liquid/alpha_adv_deciles/factors (Group) u''.

Process finished with exit code 1

Frames with 1402 columns saved just fine. Are there any limitations for the number of columns?

Metadata

Metadata

Assignees

No one assigned

    Labels

    IO HDF5read_hdf, HDFStorePerformanceMemory or execution speed performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0