-
Notifications
You must be signed in to change notification settings - Fork 874
Description
Hello @roji
The API of NpgsqlLargeObjectManager is not conducive to bulk operations. It executes too many roundtrips, one roundtrip to open a handle, then one roundtrip per chunk, then one more to close.
If I need to bulk insert 1000 products with 3000 images each stored as a large object, this takes ages even on localhost due to the large amount of roundtrips.
This also needs to be done in 1 transaction, so using multiple parallel connections is not an option. The idea I have, is some kind of thing where I can open 1000 handles at once. And send chunks of the 1000 files in a single round one lo_write to each file handle, then close them all at once. To reduce the roundtrips or something like that.
I could probably implement something like this by hand but it would be a bit awkward. Perhaps when I get some time I can do it and contribute it to Npgsql. This issue is an enhancement request for consideration.
Thanks once more for the amazing reliable work you've done all these years.