ENH: Tempfile context manager #6866
Conversation
Context manager intended for use when the same temporary file needs to be opened and closed more than once. The context manager creates the file, closes it, and returns the path to the file. On exit from the context block the file is removed. The file should be closed before exiting the context as an error will be raised on windows if not. Also fix up the `tempdir` context manager to deal with exceptions. Tests are added for both `temppath` and `tempdir`.
The test is in numpy/lib/tests/test_io.py. This commit is intended as a demonstration of using temppath.
ae085ad to
c4156cf
Compare
|
This is needed by several other PRs, so merging. |
|
That context manager yields a file path but not a file descriptor. |
|
But that file can only be opened once, and is already open. It is probably best to leave the location of the temp files up to the system administrator. It is not unusual for it to be in memory, but then, it may run out of space... |
|
I think we should use the same interface for all tempfile, whether in-mem or in-disk (see #6540) whether deleted on close or not. A lot of utils can directly work on fid (thus memfile, like zip), and it can be much much much faster this way than writing tempfile on the disk. At least for np.savez, my purpose is to make it user choice even if in-disk is the default choice. |
|
The problem come from the fact that the base class _TemporaryFileWrapper from python tempfile module Please note the choice of python to only return fid not path even for NamedTemporaryFile context manager. So all tempfile context manager have the same interface and can be transparently replace by io.BytesIO. Moreover, If i read the comments of tempfile python module, there can be an automatic deletion by the winnt system on closing tempfile, that will be a pity in our case. As I do not have WinNT I can not check it; |
Context manager intended for use when the same temporary file needs to be opened and closed more than once. The context manager creates the file, closes it, and returns the path to the file. On exit from the context block the file is removed. The file should be closed before
exiting the context as an error will be raised on windows if not.
Also fix up the
tempdircontext manager to deal with exceptions.Tests are added for both
temppathandtempdirand test_not_closing_opened_fid is refactored as an example.