8000 Reduce memory consumption inside inheritance_planner(). · postgrespro/postgres@d1001a7 · GitHub
[go: up one dir, main page]

Skip to content
  • Commit d1001a7

    Browse files
    committed
    Reduce memory consumption inside inheritance_planner().
    Avoid eating quite so much memory for large inheritance trees, by reclaiming the space used by temporary copies of the original parsetree and range table, as well as the workspace needed during planning. The cost is needing to copy the finished plan trees out of the child memory context. Although this looks like it ought to slow things down, my testing shows it actually is faster, apparently because fewer interactions with malloc() are needed and/or we can do the work within a more readily cacheable amount of memory. That result might be platform-dependent, but I'll take it. Per a gripe from John Papandriopoulos, in which it was pointed out that the memory consumption actually grew as O(N^2) for sufficiently many child tables, since we were creating N copies of the N-element range table.
    1 parent d1f5a92 commit d1001a7

    File tree

    1 file changed

    +53
    -15
    lines changed

    1 file changed

    +53
    -15
    lines changed

    src/backend/optimizer/plan/planner.c

    Lines changed: 53 additions & 15 deletions
    Original file line numberDiff line numberDiff line change
    @@ -41,6 +41,7 @@
    4141
    #include "parser/parse_oper.h"
    4242
    #include "parser/parsetree.h"
    4343
    #include "utils/lsyscache.h"
    44+
    #include "utils/memutils.h"
    4445
    #include "utils/syscache.h"
    4546

    4647

    @@ -741,19 +742,52 @@ inheritance_planner(PlannerInfo *root)
    741742
    List *rowMarks;
    742743
    List *tlist;
    743744
    PlannerInfo subroot;
    745+
    MemoryContext childcxt;
    744746
    ListCell *l;
    745747

    748+
    /*
    749+
    * Memory management here is a bit messy, because the planning process can
    750+
    * scribble on both the query parsetree and the rangetable. We need to
    751+
    * ensure that each call of grouping_planner gets an un-scribbled-on copy
    752+
    * to start with, so we start by copying the given query tree each time
    753+
    * (as a byproduct of adjust_appendrel_attrs). However, we also want to
    754+
    * make sure that the modified rangetable ultimately gets propagated back
    755+
    * to the master copy, to pick up any changes of the Query structures
    756+
    * inside subquery RTEs. We do that by copying back the rangetable from
    757+
    * the first successful child planning step. (We are effectively assuming
    758+
    * that sub-Queries will get planned identically each time, or at least
    759+
    * that the impacts on their rangetables will be the same each time.)
    760+
    *
    761+
    * Another consideration is that when there are a lot of child tables, we
    762+
    * can eat a lot of memory this way. To fix that, we create a child
    763+
    * memory context that can be reset between steps to recover memory, at
    764+
    * the cost of having to copy the completed plan trees out to the parent
    765+
    * context.
    766+
    */
    767+
    childcxt = AllocSetContextCreate(CurrentMemoryContext,
    768+
    "Inheritance child planning context",
    769+
    ALLOCSET_DEFAULT_MINSIZE,
    770+
    ALLOCSET_DEFAULT_INITSIZE,
    771+
    ALLOCSET_DEFAULT_MAXSIZE);
    772+
    746773
    foreach(l, root->append_rel_list)
    747774
    {
    748775
    AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
    749776
    Plan *subplan;
    777+
    MemoryContext oldcxt;
    750778

    751779
    /* append_rel_list contains all append rels; ignore others */
    752780
    if (appinfo->parent_relid != parentRTindex)
    753781
    continue;
    754782

    755783
    /*
    756-
    * Generate modified query with this rel as target.
    784+
    * Discard any cruft generated in previous loop iterations.
    785+
    */
    786+
    MemoryContextReset(childcxt);
    787+
    oldcxt = MemoryContextSwitchTo(childcxt);
    788+
    789+
    /*
    790+
    * Generate modified query (in childcxt) with this rel as target.
    757791
    */
    758792
    memcpy(&subroot, root, sizeof(PlannerInfo));
    759793
    subroot.parse = (Query *)
    @@ -767,24 +801,32 @@ inheritance_planner(PlannerInfo *root)
    767801
    /* and we haven't created PlaceHolderInfos, either */
    768802
    Assert(subroot.placeholder_list == NIL);
    769803

    770-
    /* Generate plan */
    804+
    /* Generate plan (also in childcxt) */
    771805
    subplan = grouping_planner(&subroot, 0.0 /* retrieve all tuples */ );
    772806

    807+
    MemoryContextSwitchTo(oldcxt);
    808+
    773809
    /*
    774810
    * If this child rel was excluded by constraint exclusion, exclude it
    775811
    * from the plan.
    776812
    */
    777813
    if (is_dummy_plan(subplan))
    778814
    continue;
    779815

    780-
    /* Save rtable from first rel for use below */
    816+
    /*
    817+
    * Be sure to copy what we need out of the child context.
    818+
    */
    819+
    subplan = copyObject(subplan);
    820+
    821+
    /* Save rtable from first child to install in parent after the loop */
    781822
    if (subplans == NIL)
    782-
    rtable = subroot.parse->rtable;
    823+
    rtable = copyObject(subroot.parse->rtable);
    783824

    784825
    subplans = lappend(subplans, subplan);
    785826

    786827
    /* Make sure any initplans from this rel get into the outer list */
    787-
    root->init_plans = list_concat(root->init_plans, subroot.init_plans);
    828+
    root->init_plans = list_concat(root->init_plans,
    829+
    copyObject(subroot.init_plans));
    788830

    789831
    /* Build target-relations list for the executor */
    790832
    resultRelations = lappend_int(resultRelations, appinfo->child_relid);
    @@ -794,14 +836,18 @@ inheritance_planner(PlannerInfo *root)
    794836
    {
    795837
    List *rlist;
    796838

    839+
    rlist = copyObject(subroot.parse->returningList);
    797840
    rlist = set_returning_clause_references(root->glob,
    798-
    subroot.parse->returningList,
    841+
    rlist,
    799842
    subplan,
    800843
    appinfo->child_relid);
    801844
    returningLists = lappend(returningLists, rlist);
    802845
    }
    803846
    }
    804847

    848+
    /* Done with child context */
    849+
    MemoryContextDelete(childcxt);
    850+
    805851
    root->resultRelations = resultRelations;
    806852

    807853
    /* Mark result as unordered (probably unnecessary) */
    @@ -824,15 +870,7 @@ inheritance_planner(PlannerInfo *root)
    824870
    }
    825871

    826872
    /*
    827-
    * Planning might have modified the rangetable, due to changes of the
    828-
    * Query structures inside subquery RTEs. We have to ensure that this
    829-
    * gets propagated back to the master copy. But can't do this until we
    830-
    * are done planning, because all the calls to grouping_planner need
    831-
    * virgin sub-Queries to work from. (We are effectively assuming that
    832-
    * sub-Queries will get planned identically each time, or at least that
    833-
    * the impacts on their rangetables will be the same each time.)
    834-
    *
    835-
    * XXX should clean this up someday
    873+
    * Install modified rangetable from first child into parent parsetree.
    836874
    */
    837875
    parse->rtable = rtable;
    838876

    0 commit comments

    Comments
     (0)
    0