FFFF Need NpgsqlLargeObjectManager but batch/bulk mode · Issue #4755 · npgsql/npgsql · GitHub
[go: up one dir, main page]

Skip to content

Need NpgsqlLargeObjectManager but batch/bulk mode #4755

@rwasef1830

Description

@rwasef1830

Hello @roji
The API of NpgsqlLargeObjectManager is not conducive to bulk operations. It executes too many roundtrips, one roundtrip to open a handle, then one roundtrip per chunk, then one more to close.

If I need to bulk insert 1000 products with 3000 images each stored as a large object, this takes ages even on localhost due to the large amount of roundtrips.

This also needs to be done in 1 transaction, so using multiple parallel connections is not an option. The idea I have, is some kind of thing where I can open 1000 handles at once. And send chunks of the 1000 files in a single round one lo_write to each file handle, then close them all at once. To reduce the roundtrips or something like that.

I could probably implement something like this by hand but it would be a bit awkward. Perhaps when I get some time I can do it and contribute it to Npgsql. This issue is an enhancement request for consideration.

Thanks once more for the amazing reliable work you've done all these years.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0