8000 recarray fromfile doesn't support file like objects (Trac #1911) · Issue #2504 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

recarray fromfile doesn't support file like objects (Trac #1911) #2504

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
numpy-gitbot opened this issue Oct 19, 2012 · 7 comments · Fixed by #16675
Closed

recarray fromfile doesn't support file like objects (Trac #1911) #2504

numpy-gitbot opened this issue Oct 19, 2012 · 7 comments · Fixed by #16675
Labels
01 - Enhancement component: numpy._core Patch

Comments

@numpy-gitbot
Copy link

Original ticket http://projects.scipy.org/numpy/ticket/1911 on 2011-07-18 by trac user sbreuss, assigned to unknown.

The current implementation of the record array fromfile method doesn't support file like objects (e.g. StringIO, io.BytesIO, ...)

By two minor changes in the source this would be possible:

  1. change the get_remaining_size method to use seek
  2. not using the readinto method of the file object

I'm open to any suggestions, a patch for the current master version showing my changes is attached.

Cheers,
Simon Breuss
@numpy-gitbot
Copy link
Author

Attachment added by trac user sbreuss on 2011-07-18: records.py.diff

@aarchiba
Copy link
Contributor

Note that even for simple float arrays text fromfile requires an actual file descriptor, so it doesn't work with StringIO.
8000

@charris
Copy link
Member
charris commented Aug 26, 2015

Is this also related to #6246?

@aarchiba
Copy link
Contributor

I don't think so. I think this is an issue about record arrays, binary files, and seeking, so there's no reason it shouldn't be fixable to work with BytesIO.

@mcmtroffaes
Copy link
Contributor

The following bug looks to be related: fromfile can take an io class object on Python 3, but not on Python 2. Specifically, the following code runs perfectly on Python 3.4, but produces an IOError on Python 2.7. I'm using numpy 1.9.2. Originally reported at jazzband/pathlib2#7 by @scienceopen

import numpy as np
import io
np.array([1.,2.,3.]).tofile("test.dat")
with io.open("test.dat", mode='rb') as f:
    print(np.fromfile(f))

EDIT: also see #6632

@kurtqq
Copy link
kurtqq commented Mar 11, 2019

Is there intention to fix this?

@eric-wieser
Copy link
Member

Another change needed here - isfileobj would need to become hasattr(fd, 'readinto'), which is more consistent with some other parts of numpy.

Supporting io.StringIO is a bad idea, only io.BytesIO should be supported.

@sidhant007 sidhant007 mentioned this issue Jun 24, 2020
Merged
mattip pushed a commit to sidhant007/numpy that referenced this issue Aug 10, 2020
@sidhant007 @mattip
ENH: Add file like support to np.core.records.fromfile (numpy#2504)
645465c
mattip pushed a commit that referenced this issue Aug 13, 2020
@sidhant007 @rossbar
ENH: Add support for file like objects to np.core.records.fromfile (#…
0311c93 
…16675)

* ENH: Add file like support to np.core.records.fromfile (#2504)

Co-authored-by: Ross Barnowski <rossbar@berkeley.edu>
@embray embray mentioned this issue Jul 20, 2021
Merged
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
01 - Enhancement component: numpy._core Patch
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ENH: Add support for file like objects to np.core.records.fromfile sidhant007/numpy
7 participants
@charris @aarchiba @mcmtroffaes @eric-wieser @mattip @numpy-gitbot @kurtqq
0