10000 Document LP_DEAD accounting issues in VACUUM. · postgrespro/postgres@7136bf3 · GitHub
[go: up one dir, main page]
Skip to content

Commit 7136bf3

Browse files
petergeogheganpetergeoghegan
committed
Document LP_DEAD accounting issues in VACUUM.
Document VACUUM's soft assumption that any LP_DEAD items encountered during pruning will become LP_UNUSED items before VACUUM finishes up. This is integral to the accounting used by VACUUM to generate its final report on the table to the stats collector. It also affects how VACUUM determines which heap pages are truncatable. In both cases VACUUM is concerned with the likely contents of the page in the near future, not the current contents of the page. This state of affairs created the false impression that VACUUM's dead tuple accounting had significant difference with similar accounting used during ANALYZE. There were and are no substantive differences, at least when the soft assumption completely works out. This is far clearer now. Also document cases where things don't quite work out for VACUUM's dead tuple accounting. It's possible that a significant number of LP_DEAD items will be left behind by VACUUM, and won't be recorded as remaining dead tuples in VACUUM's statistics collector report. This behavior dates back to commit a96c41f, which taught VACUUM to run without index and heap vacuuming at the user's request. The failsafe mechanism added to VACUUM more recently by commit 1e55e7d takes the same approach to dead tuple accounting. Reported-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-Wz=Jmtu18PrsYq3EvvZJGOmZqSO2u3bvKpx9xJa5uhNp=Q@mail.gmail.com
1 parent 640b91c commit 7136bf3

File tree

1 file changed

+42
-13
lines changed

1 file changed

+42
-13
lines changed

‎src/backend/access/heap/vacuumlazy.c

Lines changed: 42 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -686,7 +686,16 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
686686
new_min_multi,
687687
false);
688688

689-
/* report results to the stats collector, too */
689+
/*
690+
* Report results to the stats collector, too.
691+
*
692+
* Deliberately avoid telling the stats collector about LP_DEAD items that
693+
* remain in the table due to VACUUM bypassing index and heap vacuuming.
694+
* ANALYZE will consider the remaining LP_DEAD items to be dead tuples.
695+
* It seems like a good idea to err on the side of not vacuuming again too
696+
* soon in cases where the failsafe prevented significant amounts of heap
697+
* vacuuming.
698+
*/
690699
pgstat_report_vacuum(RelationGetRelid(rel),
691700
rel->rd_rel->relisshared,
692701
Max(new_live_tuples, 0),
@@ -1334,6 +1343,9 @@ lazy_scan_heap(LVRelState *vacrel, VacuumParams *params, bool aggressive)
13341343
*/
13351344
lazy_scan_prune(vacrel, buf, blkno, page, vistest, &prunestate);
13361345

1346+
Assert(!prunestate.all_visible || !prunestate.has_lpdead_items);
1347+
Assert(!all_visible_according_to_vm || prunestate.all_visible);
1348+
13371349
/* Remember the location of the last page with nonremovable tuples */
13381350
if (prunestate.hastup)
13391351
vacrel->nonempty_pages = blkno + 1;
@@ -1404,7 +1416,6 @@ lazy_scan_heap(LVRelState *vacrel, VacuumParams *params, bool aggressive)
14041416
* Handle setting visibility map bit based on what the VM said about
14051417
* the page before pruning started, and using prunestate
14061418
*/
1407-
Assert(!prunestate.all_visible || !prunestate.has_lpdead_items);
14081419
if (!all_visible_according_to_vm && prunestate.all_visible)
14091420
{
14101421
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
@@ -1786,6 +1797,14 @@ lazy_scan_prune(LVRelState *vacrel,
17861797
* The logic here is a bit simpler than acquire_sample_rows(), as
17871798
* VACUUM can't run inside a transaction block, which makes some cases
17881799
* impossible (e.g. in-progress insert from the same transaction).
1800+
*
1801+
* We treat LP_DEAD items a little differently, too -- we don't count
1802+
* them as dead_tuples at all (we only consider new_dead_tuples). The
1803+
* outcome is no different because we assume that any LP_DEAD items we
1804+
* encounter here will become LP_UNUSED inside lazy_vacuum_heap_page()
1805+
* before we report anything to the stats collector. (Cases where we
1806+
* bypass index vacuuming will violate our assumption, but the overall
1807+
* impact of that should be negligible.)
17891808
*/
17901809
switch (res)
17911810
{
@@ -1901,9 +1920,6 @@ lazy_scan_prune(LVRelState *vacrel,
19011920
* that will need to be vacuumed in indexes later, or a LP_NORMAL tuple
19021921
* that remains and needs to be considered for freezing now (LP_UNUSED and
19031922
* LP_REDIRECT items also remain, but are of no further interest to us).
1904-
*
1905-
* Add page level counters to caller's counts, and then actually process
1906-
* LP_DEAD and LP_NORMAL items.
19071923
*/
19081924
vacrel->offnum = InvalidOffsetNumber;
19091925

@@ -1988,13 +2004,6 @@ lazy_scan_prune(LVRelState *vacrel,
19882004
}
19892005
#endif
19902006

1991-
/* Add page-local counts to whole-VACUUM counts */
1992-
vacrel->tuples_deleted += tuples_deleted;
1993-
vacrel->lpdead_items += lpdead_items;
1994-
vacrel->new_dead_tuples += new_dead_tuples;
1995-
vacrel->num_tuples += num_tuples;
1996-
vacrel->live_tuples += live_tuples;
1997-
19982007
/*
19992008
* Now save details of the LP_DEAD items from the page in the dead_tuples
20002009
* array. Also record that page has dead items in per-page prunestate.
@@ -2021,6 +2030,13 @@ lazy_scan_prune(LVRelState *vacrel,
20212030
pgstat_progress_update_param(PROGRESS_VACUUM_NUM_DEAD_TUPLES,
20222031
dead_tuples->num_tuples);
20232032
}
2033+
2034+
/* Finally, add page-local counts to whole-VACUUM counts */
2035+
vacrel->tuples_deleted += tuples_deleted;
2036+
vacrel->lpdead_items += lpdead_items;
2037+
vacrel->new_dead_tuples += new_dead_tuples;
2038+
vacrel->num_tuples += num_tuples;
2039+
vacrel->live_tuples += live_tuples;
20242040
}
20252041

20262042
/*
@@ -2095,6 +2111,14 @@ lazy_vacuum(LVRelState *vacrel, bool onecall)
20952111
* not exceed 32MB. This limits the risk that we will bypass index
20962112
* vacuuming again and again until eventually there is a VACUUM whose
20972113
* dead_tuples space is not CPU cache resident.
2114+
*
2115+
* We don't take any special steps to remember the LP_DEAD items (such
2116+
* as counting them in new_dead_tuples report to the stats collector)
2117+
* when the optimization is applied. Though the accounting used in
2118+
* analyze.c's acquire_sample_rows() will recognize the same LP_DEAD
2119+
* items as dead rows in its own stats collector report, that's okay.
2120+
* The discrepancy should be negligible. If this optimization is ever
2121+
* expanded to cover more cases then this may need to be reconsidered.
20982122
*/
20992123
threshold = (double) vacrel->rel_pages * BYPASS_THRESHOLD_PAGES;
21002124
do_bypass_optimization =
@@ -2146,7 +2170,8 @@ lazy_vacuum(LVRelState *vacrel, bool onecall)
21462170
}
21472171

21482172
/*
2149-
* Forget the now-vacuumed tuples -- just press on
2173+
* Forget the LP_DEAD items that we just vacuumed (or just decided to not
2174+
* vacuum)
21502175
*/
21512176
vacrel->dead_tuples->num_tuples = 0;
21522177
}
@@ -3101,6 +3126,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
31013126
*
31023127
* Also don't attempt it if wraparound failsafe is in effect. It's hard to
31033128
* predict how long lazy_truncate_heap will take. Don't take any chances.
3129+
* There is very little chance of truncation working out when the failsafe is
3130+
* in effect in any case. lazy_scan_prune makes the optimistic assumption
3131+
* that any LP_DEAD items it encounters will always be LP_UNUSED by the time
3132+
* we're called.
31043133
*
31053134
* Also don't attempt it if we are doing early pruning/vacuuming, because a
31063135
* scan which cannot find a truncated heap page cannot determine that the

0 commit comments

Comments
 (0)
0