8000 Allow specifying ReadOnlySequence<byte> for JSON documents to avoid temporairly memory allocations by martinmine · Pull Request #4249 · npgsql/npgsql · GitHub
[go: up one dir, main page]
Skip to content

Allow specifying ReadOnlySequence<byte> for JSON documents to avoid temporairly memory allocations#4249

Closed
martinmine wants to merge 2 commits intonpgsql:mainfrom
martinmine:perf-tuning
Closed

Allow specifying ReadOnlySequence<byte> for JSON documents to avoid temporairly memory allocations#4249
martinmine wants to merge 2 commits intonpgsql:mainfrom
martinmine:perf-tuning

Conversation

@martinmine
Copy link
@martinmine martinmine commented Dec 28, 2021

This is the change in #4248 taken one step further which allows me to specify the JSONDocument as ReadOnlySequence instead of a byte[]. I can then insert many documents without any pressure on the GC:

foreach (var document in documents)
{
    using var stream = (RecyclableMemoryStream)RecyclableMemoryStreamManager.GetStream();

    if (utf8JsonWriter == null)
        utf8JsonWriter = new Utf8JsonWriter((MemoryStream)stream);
    else
        utf8JsonWriter.Reset((MemoryStream)stream);

    jsonDocument.WriteTo(utf8JsonWriter);
    utf8JsonWriter.Flush();

    var memory = stream.GetReadOnlySequence();

    using var cmd = new NpgsqlCommand("INSERT INTO mydocuments (id, document) VALUES (@document)", conn);
    cmd.Parameters.Add(new NpgsqlParameter("document", NpgsqlDbType.Jsonb)
    {
        Value = jsonDocument,
        ConvertedValue = memory
    });

    cmd.ExecuteNonQuery();
}

I am also discussing this implementation here: #1727 (comment)

@NinoFloris
Copy link
Member
NinoFloris commented Oct 3, 2023

Similar to #4248 (comment) this code doesn't exist anymore. We've already made some improvements and will be making more in this area!

@NinoFloris NinoFloris closed this Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@roji roji Awaiting requested review from roji roji is a code owner

@vonzshik vonzshik Awaiting requested review from vonzshik vonzshik is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@martinmine @NinoFloris
0