8000 Eventual error when converting svs to tiff · Issue #539 · libvips/pyvips · GitHub
[go: up one dir, main page]
Skip to content

Eventual error when converting svs to tiff #539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Tomaster134 opened this issue May 27, 2025 · 10 comments
Open

Eventual error when converting svs to tiff #539

Tomaster134 opened this issue May 27, 2025 · 10 comments

Comments

@Tomaster134
Copy link
Tomaster134 commented May 27, 2025 •
edited
Loading

Windows 10
Python 3.12.2
libvips 8.16 [all]
64gb RAM

Using John's method detailed here, I was successfully converting a 469MB .svs file into a .tiff format. I was fiddling with settings and continuously deleting and running the same snippet of code to see if I could get it to where I wanted it, when suddenly it started converting to grayscale, and then after a few more tries it started tossing out these errors. I haven't been able to determine what the cause is. Am I missing something obvious here?

Code

import os
vipsbin = r'E:\Home\Tomaster\vips-dev-8.16\bin'
add_dll_dir = getattr(os, 'add_dll_directory', None)
if callable(add_dll_dir):
    add_dll_dir(vipsbin)
else:
    os.environ['PATH'] = os.pathsep.join((vipsbin, os.environ['PATH']))


import pyvips

im = pyvips.Image.new_from_file("test/test.svs", attach_associated=True, rgb=True)

image_height = im.height

# split to separate image planes and stack vertically ready for OME 
im = pyvips.Image.arrayjoin(im.bandsplit(), across=1)

# set minimal OME metadata
# before we can modify an image (set metadata in this case), we must take a 
# private copy
im = im.copy()
im.set_type(pyvips.GValue.gint_type, "page-height", image_height)
im.set_type(pyvips.GValue.gstr_type, "image-description",
f"""<?xml version="1.0" encoding="UTF-8"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
    <Image ID="Image:0">
        <!-- Minimum required fields about image dimensions -->
        <Pixels DimensionOrder="XYCZT"
                ID="Pixels:0"
                SizeC="3"
                SizeT="1"
                SizeX="{im.width}"
                SizeY="{image_height}"
                SizeZ="1"
                Type="uint8">
        </Pixels>
    </Image>
</OME>""")

im.write_to_file("test/tada.tiff", compression="jpeg", tile=True,
            tile_width=512, tile_height=512,
            pyramid=True, subifd=True, bigtiff=True)

Error

Traceback (most recent call last):
File "E:\Home\Tomaster\Documents\tiff_co\svs_to_tiff.py", line 44, in
im.write_to_file("test/tada.tiff", compression="jpeg", tile=True,
File "E:\Home\Tomaster\Documents\tiff_co\venv\Lib\site-packages\pyvips\vimage.py", line 781, in write_to_file
return pyvips.Operation.call(name, self, filename,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\Home\Tomaster\Documents\tiff_co\venv\Lib\site-packages\pyvips\voperation.py", line 301, in call
raise Error(f'unable to call {operation_name}')
pyvips.error.Error: unable to call VipsForeignSaveTiffFile
openslide2vips: reading region: Couldn't open test/test.svs: Too many open files
test/test.svs: Cannot read TIFF header
openslide2vips: reading region: Couldn't open test/test.svs: Too many open files
openslide2vips: reading region: Couldn't open test/test.svs: Too many open files
...repeats ad nauseam with several hundred more errors
@jcupitt
Copy link
Member
jcupitt commented May 27, 2025

Hi @Tomaster134,

openslide2vips: reading region: Couldn't open test/test.svs: Too many open files

Windows has a limit of 1024 open files, perhaps you've somehow hit that? Restarting your python session should reset it.

@Tomaster134
Copy link
Author

I've tried killing the terminal window, deleting and rebuilding the venv, and resetting my whole computer, all to no avail. Is there something else that I'm missing that would be worth trying?

@jcupitt
Copy link
Member
jcupitt commented May 27, 2025

Oh dear :( Maybe the file is corrupt? Have you tried a known-good SVS? For example:

https://openslide.cs.cmu.edu/download/openslide-testdata/Aperio/CMU-1.svs

@jcupitt
Copy link
Member
jcupitt commented May 27, 2025

I'd remove attach_associated=True, I don't think you need it.

@Tomaster134
Copy link
Author

Hmm. I just ran it on the CMU-1.svs, and it didn't throw any errors, but it converted to grayscale?

Also removed the attach_associated since you are correct, it seems unnecessary.

@jcupitt
Copy link
Member
jcupitt commented May 27, 2025

What are you using to view the TIFF file? It'll need to be something like QuPath or vipsdisp that has a fancy OME-TIFF loader.

@Tomaster134
Copy link
Author

Oops! I was being lazy and going off the thumbnail, it comes out perfectly in QuPath. I guess my .svs was just corrupted in some way? I'm curious as to why it didn't start throwing error messages until a few hours in, but oh well. It also doesn't seem to throw the error messages if I'm not using the im = im.copy() to edit metadata, but maybe that's just due to the corruption?

@jcupitt
Copy link
Member
jcupitt commented May 27, 2025

I could have a look at the file if you have somewhere to upload it.

@Tomaster134
Copy link
Author

I can definitely do that, but I'd have to remove identifying information from the slide label. Would it still be useful if it's been changed in that way?

@jcupitt
Copy link
Member
jcupitt commented May 28, 2025

Sure, but you may not be able to anonymise it if the file has been truncated :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jcupitt @Tomaster134
0