8000 Enable parallel query with SERIALIZABLE isolation. · postgres/postgres@bb16aba · GitHub
[go: up one dir, main page]
Skip to content

Commit bb16aba

Browse files
macdicemacdice
committed
Enable parallel query with SERIALIZABLE isolation.
Previously, the SERIALIZABLE isolation level prevented parallel query from being used. Allow the two features to be used together by sharing the leader's SERIALIZABLEXACT with parallel workers. An extra per-SERIALIZABLEXACT LWLock is introduced to make it safe to share, and new logic is introduced to coordinate the early release of the SERIALIZABLEXACT required for the SXACT_FLAG_RO_SAFE optimization, as follows: The first backend to observe the SXACT_FLAG_RO_SAFE flag (set by some other transaction) will 'partially release' the SERIALIZABLEXACT, meaning that the conflicts and locks it holds are released, but the SERIALIZABLEXACT itself will remain active because other backends might still have a pointer to it. Whenever any backend notices the SXACT_FLAG_RO_SAFE flag, it clears its own MySerializableXact variable and frees local resources so that it can skip SSI checks for the rest of the transaction. In the special case of the leader process, it transfers the SERIALIZABLEXACT to a new variable SavedSerializableXact, so that it can be completely released at the end of the transaction after all workers have exited. Remove the serializable_okay flag added to CreateParallelContext() by commit 9da0cc3, because it's now redundant. Author: Thomas Munro Reviewed-by: Haribabu Kommi, Robert Haas, Masahiko Sawada, Kevin Grittner Discussion: https://postgr.es/m/CAEepm=0gXGYhtrVDWOTHS8SQQy_=S9xo+8oCxGLWZAOoeJ=yzQ@mail.gmail.com
1 parent 13e8643 commit bb16aba

File tree

19 files changed

+429
-67
lines changed
  • isolation_schedule
  • specs

    • 19 files changed

    +429
    -67
    lines changed

    ‎doc/src/sgml/monitoring.sgml

    Lines changed: 6 additions & 1 deletion
    Original file line numberDiff line numberDiff line change
    @@ -861,7 +861,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
    861861

    862862
    <tbody>
    863863
    <row>
    864-
    <entry morerows="63"><literal>LWLock</literal></entry>
    864+
    <entry morerows="64"><literal>LWLock</literal></entry>
    865865
    <entry><literal>ShmemIndexLock</literal></entry>
    866866
    <entry>Waiting to find or allocate space in shared memory.</entry>
    867867
    </row>
    @@ -1121,6 +1121,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
    11211121
    <entry><literal>predicate_lock_manager</literal></entry>
    11221122
    <entry>Waiting to add or examine predicate lock information.</entry>
    11231123
    </row>
    1124+
    <row>
    1125+
    <entry><literal>serializable_xact</literal></entry>
    1126+
    <entry>Waiting to perform an operation on a serializable transaction
    1127+
    in a parallel query.</entry>
    1128+
    </row>
    11241129
    <row>
    11251130
    <entry><literal>parallel_query_dsa</literal></entry>
    11261131
    <entry>Waiting for parallel query dynamic shared memory allocation lock.</entry>

    ‎doc/src/sgml/parallel.sgml

    Lines changed: 0 additions & 17 deletions
    Original file line numberDiff line numberDiff line change
    @@ -184,13 +184,6 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
    184184
    using a very large number of processes.
    185185
    </para>
    186186
    </listitem>
    187-
    188-
    <listitem>
    189-
    <para>
    190-
    The transaction isolation level is serializable. This is
    191-
    a limitation of the current implementation.
    192-
    </para>
    193-
    </listitem>
    194187
    </itemizedlist>
    195188

    196189
    <para>
    @@ -233,16 +226,6 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
    233226
    that may be suboptimal when run serially.
    234227
    </para>
    235228
    </listitem>
    236-
    237-
    <listitem>
    238-
    <para>
    239-
    The transaction isolation level is serializable. This situation
    240-
    does not normally arise, because parallel query plans are not
    241-
    generated when the transaction isolation level is serializable.
    242-
    However, it can happen if the transaction isolation level is changed to
    243-
    serializable after the plan is generated and before it is executed.
    244-
    </para>
    245-
    </listitem>
    246229
    </itemizedlist>
    247230
    </sect1>
    248231

    ‎src/backend/access/nbtree/nbtsort.c

    Lines changed: 1 addition & 1 deletion
    Original file line numberDiff line numberDiff line change
    @@ -1265,7 +1265,7 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
    12651265
    EnterParallelMode();
    12661266
    Assert(request > 0);
    12671267
    pcxt = CreateParallelContext("postgres", "_bt_parallel_build_main",
    1268-
    request, true);
    1268+
    request);
    12691269
    scantuplesortstates = leaderparticipates ? request + 1 : request;
    12701270

    12711271
    /*

    ‎src/backend/access/transam/parallel.c

    Lines changed: 7 additions & 11 deletions
    Original file line numberDiff line numberDiff line change
    @@ -31,6 +31,7 @@
    3131
    #include "optimizer/optimizer.h"
    3232
    #include "pgstat.h"
    3333
    #include "storage/ipc.h"
    34+
    #include "storage/predicate.h"
    3435
    #include "storage/sinval.h"
    3536
    #include "storage/spin.h"
    3637
    #include "tcop/tcopprot.h"
    @@ -91,6 +92,7 @@ typedef struct FixedParallelState
    9192
    BackendId parallel_master_backend_id;
    9293
    TimestampTz xact_ts;
    9394
    TimestampTz stmt_ts;
    95+
    SerializableXactHandle serializable_xact_handle;
    9496

    9597
    /* Mutex protects remaining fields. */
    9698
    slock_t mutex;
    @@ -155,7 +157,7 @@ static void ParallelWorkerShutdown(int code, Datum arg);
    155157
    */
    156158
    ParallelContext *
    157159
    CreateParallelContext(const char *library_name, const char *function_name,
    158-
    int nworkers, bool serializable_okay)
    160+
    int nworkers)
    159161
    {
    160162
    MemoryContext oldcontext;
    161163
    ParallelContext *pcxt;
    @@ -166,16 +168,6 @@ CreateParallelContext(const char *library_name, const char *function_name,
    166168
    /* Number of workers should be non-negative. */
    167169
    Assert(nworkers >= 0);
    168170

    169-
    /*
    170-
    * If we are running under serializable isolation, we can't use parallel
    171-
    * workers, at least not until somebody enhances that mechanism to be
    172-
    * parallel-aware. Utility statement callers may ask us to ignore this
    173-
    * restriction because they're always able to safely ignore the fact that
    174-
    * SIREAD locks do not work with parallelism.
    175-
    */
    176-
    if (IsolationIsSerializable() && !serializable_okay)
    177-
    nworkers = 0;
    178-
    179171
    /* We might be running in a short-lived memory context. */
    180172
    oldcontext = MemoryContextSwitchTo(TopTransactionContext);
    181173

    @@ -327,6 +319,7 @@ InitializeParallelDSM(ParallelContext *pcxt)
    327319
    fps->parallel_master_backend_id = MyBackendId;
    328320
    fps->xact_ts = GetCurrentTransactionStartTimestamp();
    329321
    fps->stmt_ts = GetCurrentStatementStartTimestamp();
    322+
    fps->serializable_xact_handle = ShareSerializableXact();
    330323
    SpinLockInit(&fps->mutex);
    331324
    fps->last_xlog_end = 0;
    332325
    shm_toc_insert(pcxt->toc, PARALLEL_KEY_FIXED, fps);
    @@ -1422,6 +1415,9 @@ ParallelWorkerMain(Datum main_arg)
    14221415
    false);
    14231416
    RestoreEnumBlacklist(enumblacklistspace);
    14241417

    1418+
    /* Attach to the leader's serializable transaction, if SERIALIZABLE. */
    1419+
    AttachSerializableXact(fps->serializable_xact_handle);
    1420+
    14251421
    /*
    14261422
    * We've initialized all of our state now; nothing should change
    14271423
    * hereafter.

    ‎src/backend/access/transam/xact.c

    Lines changed: 5 additions & 2 deletions
    Original file line numberDiff line numberDiff line change
    @@ -2024,9 +2024,12 @@ CommitTransaction(void)
    20242024
    /*
    20252025
    * Mark serializable transaction as complete for predicate locking
    20262026
    * purposes. This should be done as late as we can put it and still allow
    2027-
    * errors to be raised for failure patterns found at commit.
    2027+
    * errors to be raised for failure patterns found at commit. This is not
    2028+
    * appropriate in a parallel worker however, because we aren't committing
    2029+
    * the leader's transaction and its serializable state will live on.
    20282030
    */
    2029-
    PreCommit_CheckForSerializationFailure();
    2031+
    if (!is_parallel_worker)
    2032+
    PreCommit_CheckForSerializationFailure();
    20302033

    20312034
    /*
    20322035
    * Insert notifications sent by NOTIFY commands into the queue. This

    ‎src/backend/executor/execParallel.c

    Lines changed: 1 addition & 1 deletion
    Original file line numberDiff line numberDiff line change
    @@ -604,7 +604,7 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
    604604
    pstmt_data = ExecSerializePlan(planstate->plan, estate);
    605605

    606606
    /* Create a parallel context. */
    607-
    pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers, false);
    607+
    pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers);
    608608
    pei->pcxt = pcxt;
    609609

    610610
    /*

    ‎src/backend/optimizer/plan/planner.c

    Lines changed: 1 addition & 10 deletions
    Original file line numberDiff line numberDiff line change
    @@ -337,22 +337,13 @@ standard_planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
    337337
    * parallel worker. We might eventually be able to relax this
    338338
    * restriction, but for now it seems best not to have parallel workers
    339339
    * trying to create their own parallel workers.
    340-
    *
    341-
    * We can't use parallelism in serializable mode because the predicate
    342-
    * locking code is not parallel-aware. It's not catastrophic if someone
    343-
    * tries to run a parallel plan in serializable mode; it just won't get
    344-
    * any workers and will run serially. But it seems like a good heuristic
    345-
    * to assume that the same serialization level will be in effect at plan
    346-
    * time and execution time, so don't generate a parallel plan if we're in
    347-
    * serializable mode.
    348340
    */
    349341
    if ((cursorOptions & CURSOR_OPT_PARALLEL_OK) != 0 &&
    350342
    IsUnderPostmaster &&
    351343
    parse->commandType == CMD_SELECT &&
    352344
    !parse->hasModifyingCTE &&
    353345
    max_parallel_workers_per_gather > 0 &&
    354-
    !IsParallelWorker() &&
    355-
    !IsolationIsSerializable())
    346+
    !IsParallelWorker())
    356347
    {
    357348
    /* all the cheap tests pass, so scan the query tree */
    358349
    glob->maxParallelHazard = max_parallel_hazard(parse);

    ‎src/backend/storage/lmgr/lwlock.c

    Lines changed: 1 addition & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -521,6 +521,7 @@ RegisterLWLockTranches(void)
    521521
    LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
    522522
    LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
    523523
    LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
    524+
    LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
    524525

    525526
    /* Register named tranches. */
    526527
    for (i = 0; i < NamedLWLockTrancheRequests; i++)

    0 commit comments

    Comments
     (0)
    0