8000 Make the upper part of the planner work by generating and comparing P… · prmdeveloper/postgres@3fc6e2d · GitHub
[go: up one dir, main page]
Skip to content

Commit 3fc6e2d

Browse files
tglsfdctglsfdc
committed
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it finally is. This patch removes the ever-growing tangle of spaghetti logic that grouping_planner() used to use to try to identify the best plan for post-scan/join query steps. Now, there is (nearly) independent consideration of each execution step, and entirely separate construction of Paths to represent each of the possible ways to do that step. We choose the best Path or set of Paths using the same add_path() logic that's been used inside query_planner() for years. In addition, this patch removes the old restriction that subquery_planner() could return only a single Plan. It now returns a RelOptInfo containing a set of Paths, just as query_planner() does, and the parent query level can use each of those Paths as the basis of a SubqueryScanPath at its level. This allows finding some optimizations that we missed before, wherein a subquery was capable of returning presorted data and thereby avoiding a sort in the parent level, making the overall cost cheaper even though delivering sorted output was not the cheapest plan for the subquery in isolation. (A couple of regression test outputs change in consequence of that. However, there is very little change in visible planner behavior overall, because the point of this patch is not to get immediate planning benefits but to create the infrastructure for future improvements.) There is a great deal left to do here. This patch unblocks a lot of planner work that was basically impractical in the old code structure, such as allowing FDWs to implement remote aggregation, or rewriting plan_set_operations() to allow consideration of multiple implementation orders for set operations. (The latter will likely require a full rewrite of plan_set_operations(); what I've done here is only to fix it to return Paths not Plans.) I have also left unfinished some localized refactoring in createplan.c and planner.c, because it was not necessary to get this patch to a working state. Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
1 parent b642e50 commit 3fc6e2d

35 files changed

+5573
-3108
lines changed

‎doc/src/sgml/fdwhandler.sgml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1316,6 +1316,40 @@ GetForeignServerByName(const char *name, bool missing_ok);
13161316
(<literal>extra-&gt;restrictlist</>).
13171317
</para>
13181318

1319+
<para>
1320+
An FDW might additionally support direct execution of some plan actions
1321+
that are above the level of scans and joins, such as grouping or
1322+
aggregation. To offer such options, the FDW should generate paths
1323+
(probably ForeignPaths or CustomPaths) and insert them into the
1324+
appropriate <firstterm>upper relation</>. For example, a path
1325+
representing remote aggregation should be inserted into the relation
1326+
obtained from <literal>fetch_upper_rel(root, UPPERREL_GROUP_AGG,
1327+
NULL)</>, using <function>add_path</>. This path will be compared on a
1328+
cost basis with local aggregation performed by reading a simple scan path
1329+
for the foreign relation (note that such a path must also be supplied,
1330+
else there will be an error at plan time). If the remote-aggregation
1331+
path wins, which it usually would, it will be converted into a plan in
1332+
the usual way, by calling <function>GetForeignPlan</>.
1333+
</para>
1334+
1335+
<para>
1336+
<function>PlanForeignModify</> and the other callbacks described in
1337+
<xref linkend="fdw-callbacks-update"> are designed around the assumption
1338+
that the foreign relation will be scanned in the usual way and then
1339+
individual row updates will be driven by a local <literal>ModifyTable</>
1340+
plan node. This approach is necessary for the general case where an
1341+
update requires reading local tables as well as foreign tables.
1342+
However, if the operation could be executed entirely by the foreign
1343+
server, the FDW could generate a path representing that and insert it
1344+
into the <literal>UPPERREL_FINAL</> upper relation, where it would
1345+
compete against the <literal>ModifyTable</> approach. This approach
1346+
could also be used to implement remote <literal>SELECT FOR UPDATE</>,
1347+
rather than using the row locking callbacks described in
1348+
<xref linkend="fdw-callbacks-row-locking">. Keep in mind that a path
1349+
inserted into <literal>UPPERREL_FINAL</> is responsible for
1350+
implementing <emphasis>all</> behavior of the query.
1351+
</para>
1352+
13191353
<para>
13201354
When planning an <command>UPDATE</> or <command>DELETE</>,
13211355
<function>PlanForeignModify</> can look up the <structname>RelOptInfo</>

‎src/backend/executor/execAmi.c

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -407,17 +407,20 @@ ExecSupportsMarkRestore(Path *pathnode)
407407
case T_Result:
408408

409409
/*
410-
* Although Result supports mark/restore if it has a child plan
411-
* that does, we presently come here only for ResultPath nodes,
412-
* which represent Result plans without a child plan. So there is
413-
* nothing to recurse to and we can just say "false". (This means
414-
* that Result's support for mark/restore is in fact dead code. We
415-
* keep it since it's not much code, and someday the planner might
416-
* be smart enough to use it. That would require making this
417-
* function smarter too, of course.)
410+
* Result supports mark/restore iff it has a child plan that does.
411+
*
412+
* We have to be careful here because there is more than one Path
413+
* type that can produce a Result plan node.
418414
*/
419-
Assert(IsA(pathnode, ResultPath));
420-
return false;
415+
if (IsA(pathnode, ProjectionPath))
416+
return ExecSupportsMarkRestore(((ProjectionPath *) pathnode)->subpath);
417+
else if (IsA(pathnode, MinMaxAggPath))
418+
return false; /* childless Result */
419+
else
420+
{
421+
Assert(IsA(pathnode, ResultPath));
422+
return false; /* childless Result */
423+
}
421424

422425
default:
423426
break;

‎src/backend/nodes/copyfuncs.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -867,9 +867,9 @@ _copyAgg(const Agg *from)
867867
CopyPlanFields((const Plan *) from, (Plan *) newnode);
868868

869869
COPY_SCALAR_FIELD(aggstrategy);
870-
COPY_SCALAR_FIELD(numCols);
871870
COPY_SCALAR_FIELD(combineStates);
872871
COPY_SCALAR_FIELD(finalizeAggs);
872+
COPY_SCALAR_FIELD(numCols);
873873
if (from->numCols > 0)
874874
{
875875
COPY_POINTER_FIELD(grpColIdx, from->numCols * sizeof(AttrNumber));

‎src/backend/nodes/outfuncs.c

Lines changed: 233 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -706,21 +706,19 @@ _outAgg(StringInfo str, const Agg *node)
706706
_outPlanInfo(str, (const Plan *) node);
707707

708708
WRITE_ENUM_FIELD(aggstrategy, AggStrategy);
709+
WRITE_BOOL_FIELD(combineStates);
710+
WRITE_BOOL_FIELD(finalizeAggs);
709711
WRITE_INT_FIELD(numCols);
710712

711713
appendStringInfoString(str, " :grpColIdx");
712714
for (i = 0; i < node->numCols; i++)
713715
appendStringInfo(str, " %d", node->grpColIdx[i]);
714716

715-
WRITE_BOOL_FIELD(combineStates);
716-
WRITE_BOOL_FIELD(finalizeAggs);
717-
718717
appendStringInfoString(str, " :grpOperators");
719718
for (i = 0; i < node->numCols; i++)
720719
appendStringInfo(str, " %u", node->grpOperators[i]);
721720

722721
WRITE_LONG_FIELD(numGroups);
723-
724722
WRITE_NODE_FIELD(groupingSets);
725723
WRITE_NODE_FIELD(chain);
726724
}
@@ -1603,6 +1601,15 @@ _outPathInfo(StringInfo str, const Path *node)
16031601
if (node->pathtarget != &(node->parent->reltarget))
16041602
{
16051603
WRITE_NODE_FIELD(pathtarget->exprs);
1604+
if (node->pathtarget->sortgrouprefs)
1605+
{
1606+
int i;
1607+
1608+
appendStringInfoString(str, " :pathtarget->sortgrouprefs");
1609+
for (i = 0; i < list_length(node->pathtarget->exprs); i++)
1610+
appendStringInfo(str, " %u",
1611+
node->pathtarget->sortgrouprefs[i]);
1612+
}
16061613
WRITE_FLOAT_FIELD(pathtarget->cost.startup, "%.2f");
16071614
WRITE_FLOAT_FIELD(pathtarget->cost.per_tuple, "%.2f");
16081615
WRITE_INT_FIELD(pathtarget->width);
@@ -1703,6 +1710,16 @@ _outTidPath(StringInfo str, const TidPath *node)
17031710
WRITE_NODE_FIELD(tidquals);
17041711
}
17051712

1713+
static void
1714+
_outSubqueryScanPath(StringInfo str, const SubqueryScanPath *node)
1715+
{
1716+
WRITE_NODE_TYPE("SUBQUERYSCANPATH");
1717+
1718+
_outPathInfo(str, (const Path *) node);
1719+
1720+
WRITE_NODE_FIELD(subpath);
1721+
}
1722+
17061723
static void
17071724
_outForeignPath(StringInfo str, const ForeignPath *node)
17081725
{
@@ -1793,6 +1810,174 @@ _outGatherPath(StringInfo str, const GatherPath *node)
17931810
WRITE_BOOL_FIELD(single_copy);
17941811
}
17951812

1813+
static void
1814+
_outProjectionPath(StringInfo str, const ProjectionPath *node)
1815+
{
1816+
WRITE_NODE_TYPE("PROJECTIONPATH");
1817+
1818+
_outPathInfo(str, (const Path *) node);
1819+
1820+
WRITE_NODE_FIELD(subpath);
1821+
}
1822+
1823+
static void
1824+
_outSortPath(StringInfo str, const SortPath *node)
1825+
{
1826+
WRITE_NODE_TYPE("SORTPATH");
1827+
1828+
_outPathInfo(str, (const Path *) node);
1829+
1830+
WRITE_NODE_FIELD(subpath);
1831+
}
1832+
1833+
static void
1834+
_outGroupPath(StringInfo str, const GroupPath *node)
1835+
{
1836+
WRITE_NODE_TYPE("GROUPPATH");
1837+
1838+
_outPathInfo(str, (const Path *) node);
1839+
1840+
WRITE_NODE_FIELD(subpath);
1841+
WRITE_NODE_FIELD(groupClause);
1842+
WRITE_NODE_FIELD(qual);
1843+
}
1844+
1845+
static void
1846+
_outUpperUniquePath(StringInfo str, const UpperUniquePath *node)
1847+
{
1848+
WRITE_NODE_TYPE("UPPERUNIQUEPATH");
1849+
1850+
_outPathInfo(str, (const Path *) node);
1851+
1852+
WRITE_NODE_FIELD(subpath);
1853+
WRITE_INT_FIELD(numkeys);
1854+
}
1855+
1856+
static void
1857+
_outAggPath(StringInfo str, const AggPath *node)
1858+
{
1859+
WRITE_NODE_TYPE("AGGPATH");
1860+
1861+
_outPathInfo(str, (const Path *) node);
1862+
1863+
WRITE_NODE_FIELD(subpath);
1864+
WRITE_ENUM_FIELD(aggstrategy, AggStrategy);
1865+
WRITE_FLOAT_FIELD(numGroups, "%.0f");
1866+
WRITE_NODE_FIELD(groupClause);
1867+
WRITE_NODE_FIELD(qual);
1868+
}
1869+
1870+
static void
1871+
_outGroupingSetsPath(StringInfo str, const GroupingSetsPath *node)
1872+
{
1873+
WRITE_NODE_TYPE("GROUPINGSETSPATH");
1874+
1875+
_outPathInfo(str, (const Path *) node);
1876+
1877+
WRITE_NODE_FIELD(subpath);
1878+
/* we don't bother to print groupColIdx */
1879+
WRITE_NODE_FIELD(rollup_groupclauses);
1880+
WRITE_NODE_FIELD(rollup_lists);
1881+
WRITE_NODE_FIELD(qual);
1882+
}
1883+
1884+
static void
1885+
_outMinMaxAggPath(StringInfo str, const MinMaxAggPath *node)
1886+
{
1887+
WRITE_NODE_TYPE("MINMAXAGGPATH");
1888+
1889+
_outPathInfo(str, (const Path *) node);
1890+
1891+
WRITE_NODE_FIELD(mmaggregates);
1892+
WRITE_NODE_FIELD(quals);
1893+
}
1894+
1895+
static void
1896+
_outWindowAggPath(StringInfo str, const WindowAggPath *node)
1897+
{
1898+
WRITE_NODE_TYPE("WINDOWAGGPATH");
1899+
1900+
_outPathInfo(str, (const Path *) node);
1901+
1902+
WRITE_NODE_FIELD(subpath);
1903+
WRITE_NODE_FIELD(winclause);
1904+
WRITE_NODE_FIELD(winpathkeys);
1905+
}
1906+
1907+
static void
1908+
_outSetOpPath(StringInfo str, const SetOpPath *node)
1909+
{
1910+
WRITE_NODE_TYPE("SETOPPATH");
1911+
1912+
_outPathInfo(str, (const Path *) node);
1913+
1914+
WRITE_NODE_FIELD(subpath);
1915+
WRITE_ENUM_FIELD(cmd, SetOpCmd);
1916+
WRITE_ENUM_FIELD(strategy, SetOpStrategy);
1917+
WRITE_NODE_FIELD(distinctList);
1918+
WRITE_INT_FIELD(flagColIdx);
1919+
WRITE_INT_FIELD(firstFlag);
1920+
WRITE_FLOAT_FIELD(numGroups, "%.0f");
1921+
}
1922+
1923+
static void
1924+
_outRecursiveUnionPath(StringInfo str, const RecursiveUnionPath *node)
1925+
{
1926+
WRITE_NODE_TYPE("RECURSIVEUNIONPATH");
1927+
1928+
_outPathInfo(str, (const Path *) node);
1929+
1930+
WRITE_NODE_FIELD(leftpath);
1931+
WRITE_NODE_FIELD(rightpath);
1932+
WRITE_NODE_FIELD(distinctList);
1933+
WRITE_INT_FIELD(wtParam);
1934+
WRITE_FLOAT_FIELD(numGroups, "%.0f");
1935+
}
1936+
1937+
static void
1938+
_outLockRowsPath(StringInfo str, const LockRowsPath *node)
1939+
{
1940+
WRITE_NODE_TYPE("LOCKROWSPATH");
1941+
1942+
_outPathInfo(str, (const Path *) node);
1943+
1944+
WRITE_NODE_FIELD(subpath);
1945+
WRITE_NODE_FIELD(rowMarks);
1946+
WRITE_INT_FIELD(epqParam);
1947+
}
1948+
1949+
static void
1950+
_outModifyTablePath(StringInfo str, const ModifyTablePath *node)
1951+
{
1952+
WRITE_NODE_TYPE("MODIFYTABLEPATH");
1953+
1954+
_outPathInfo(str, (const Path *) node);
1955+
1956+
WRITE_ENUM_FIELD(operation, CmdType);
1957+
WRITE_BOOL_FIELD(canSetTag);
1958+
WRITE_UINT_FIELD(nominalRelation);
1959+
WRITE_NODE_FIELD(resultRelations);
1960+
WRITE_NODE_FIELD(subpaths);
1961+
WRITE_NODE_FIELD(subroots);
1962+
WRITE_NODE_FIELD(withCheckOptionLists);
1963+
WRITE_NODE_FIELD(returningLists);
1964+
WRITE_NODE_FIELD(rowMarks);
1965+
WRITE_NODE_FIELD(onconflict);
1966+
WRITE_INT_FIELD(epqParam);
1967+
}
1968+
1969+
static void
1970+
_outLimitPath(StringInfo str, const LimitPath *node)
1971+
{
1972+
WRITE_NODE_TYPE("LIMITPATH");
1973+
1974+
_outPathInfo(str, (const Path *) node);
1975+
1976+
WRITE_NODE_FIELD(subpath);
1977+
WRITE_NODE_FIELD(limitOffset);
1978+
WRITE_NODE_FIELD(limitCount);
1979+
}
1980+
17961981
static void
17971982
_outNestPath(StringInfo str, const NestPath *node)
17981983
{
@@ -1881,6 +2066,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
18812066
WRITE_NODE_FIELD(window_pathkeys);
18822067
WRITE_NODE_FIELD(distinct_pathkeys);
18832068
WRITE_NODE_FIELD(sort_pathkeys);
2069+
WRITE_NODE_FIELD(processed_tlist);
18842070
WRITE_NODE_FIELD(minmax_aggs);
18852071
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
18862072
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -1910,6 +2096,7 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
19102096
WRITE_BOOL_FIELD(consider_param_startup);
1 D7AE 9112097
WRITE_BOOL_FIELD(consider_parallel);
19122098
WRITE_NODE_FIELD(reltarget.exprs);
2099+
/* reltarget.sortgrouprefs is never interesting, at present anyway */
19132100
WRITE_FLOAT_FIELD(reltarget.cost.startup, "%.2f");
19142101
WRITE_FLOAT_FIELD(reltarget.cost.per_tuple, "%.2f");
19152102
WRITE_INT_FIELD(reltarget.width);
@@ -1933,7 +2120,6 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
19332120
WRITE_UINT_FIELD(pages);
19342121
WRITE_FLOAT_FIELD(tuples, "%.0f");
19352122
WRITE_FLOAT_FIELD(allvisfrac, "%.6f");
1936-
WRITE_NODE_FIELD(subplan);
19372123
WRITE_NODE_FIELD(subroot);
19382124
WRITE_NODE_FIELD(subplan_params);
19392125
WRITE_OID_FIELD(serverid);
@@ -3331,6 +3517,9 @@ _outNode(StringInfo str, const void *obj)
33313517
case T_TidPath:
33323518
_outTidPath(str, obj);
33333519
break;
3520+
case T_SubqueryScanPath:
3521+
_outSubqueryScanPath(str, obj);
3522+
break;
33343523
case T_ForeignPath:
33353524
_outForeignPath(str, obj);
33363525
break;
@@ -3355,6 +3544,45 @@ _outNode(StringInfo str, const void *obj)
33553544
case T_GatherPath:
33563545
_outGatherPath(str, obj);
33573546
break;
3547+
case T_ProjectionPath:
3548+
_outProjectionPath(str, obj);
3549+
break;
3550+
case T_SortPath:
3551+
_outSortPath(str, obj);
3552+
break;
3553+
case T_GroupPath:
3554+
_outGroupPath(str, obj);
3555+
break;
3556+
case T_UpperUniquePath:
3557+
_outUpperUniquePath(str, obj);
3558+
break;
3559+
case T_AggPath:
3560+
_outAggPath(str, obj);
3561+
break;
3562+
case T_GroupingSetsPath:
3563+
_outGroupingSetsPath(str, obj);
3564+
break;
3565+
case T_MinMaxAggPath:
3566+
_outMinMaxAggPath(str, obj);
3567+
break;
3568+
case T_WindowAggPath:
3569+
_outWindowAggPath(str, obj);
3570+
break;
3571+
case T_SetOpPath:
3572+
_outSetOpPath(str, obj);
3573+
break;
3574+
case T_RecursiveUnionPath:
3575+
_outRecursiveUnionPath(str, obj);
3576+
break;
3577+
case T_LockRowsPath:
3578+
_outLockRowsPath(str, obj);
3579+
break;
3580+
case T_ModifyTablePath:
3581+
_outModifyTablePath(str, obj);
3582+
break;
3583+
case T_LimitPath:
3584+
_outLimitPath(str, obj);
3585+
break;
33583586
case T_NestPath:
33593587
_outNestPath(str, obj);
33603588
break;

0 commit comments

Comments
 (0)
0