8000 Heavily refactor btsortsupport_worker. · postgrespro/postgres@b529b65 · GitHub
[go: up one dir, main page]
Skip to content

Commit b529b65

Browse files
robertmhaasrobertmhaas
committed
Heavily refactor btsortsupport_worker.
Prior to commit 4ea51cd, this function only had one job, which was to decide whether we could avoid trampolining through the fmgr layer when performing sort comparisons. As of that commit, it has a second job, which is to decide whether we can use abbreviated keys. Unfortunately, those two tasks are somewhat intertwined in the existing coding, which is likely why neither Peter Geoghegan nor I noticed prior to commit that this calls pg_newlocale_from_collation() in cases where it didn't previously. The buildfarm noticed, though. To fix, rewrite the logic so that the decision as to which comparator to use is more cleanly separated from the decision about abbreviation.
1 parent 813ffc0 commit b529b65

File tree

1 file changed

+86
-105
lines changed

1 file changed

+86
-105
lines changed

‎src/backend/utils/adt/varlena.c

Lines changed: 86 additions & 105 deletions
Original file line numberDiff line numberDiff line change
@@ -1743,137 +1743,118 @@ bttextsortsupport(PG_FUNCTION_ARGS)
17431743
static void
17441744
btsortsupport_worker(SortSupport ssup, Oid collid)
17451745
{
1746-
TextSortSupport *tss;
17471746
bool abbreviate = ssup->abbreviate;
1747+
bool locale_aware = false;
1748+
TextSortSupport *tss;
17481749

1749-
/*
1750-
* WIN32 requires complex hacks when the database encoding is UTF-8 (except
1751-
* when using the "C" collation). For now, we don't optimize that case.
1752-
* The use of abbreviated keys is also disabled on Windows, because
1753-
* strxfrm() doesn't appear to work properly on some Windows systems.
1754-
* Ideally, we would use it on those systems where it's reliable and
1755-
* skip it only for the rest, but at the moment we don't know how to
1756-
* distinguish between the ones where it works and the ones where it
1757-
* doesn't.
1758-
*/
1759-
#ifdef WIN32
1760-
if (GetDatabaseEncoding() == PG_UTF8 && !lc_collate_is_c(collid))
1761-
return;
1762-
abbreviate = false;
1750+
#ifdef HAVE_LOCALE_T
1751+
pg_locale_t locale = 0;
17631752
#endif
17641753

17651754
/*
1766-
* On platforms where the abbreviated key for text optimization might have
1767-
* bad worst case performance, it may be useful to avoid it entirely by
1768-
* disabling it at compile time. Having only 4 byte datums could make
1769-
* worst-case performance drastically more likely, for example. Moreover,
1770-
* Darwin's strxfrm() implementations is known to not effectively
1771-
* concentrate a significant amount of entropy from the original string in
1772-
* earlier transformed blobs. It's possible that other supported platforms
1773-
* are similarly encumbered.
1774-
*
1775-
* Any reasonable implementation will pack primary weights into the start
1776-
* of returned blobs. The canonical algorithm's implementation is
1777-
* discussed by Unicode Technical Standard #10 ("UNICODE COLLATION
1778-
* ALGORITHM"), section 4, "Main algorithm". Section 4.3, "Form Sort Key"
1779-
* is of particular interest:
1780-
*
1781-
* http://www.unicode.org/reports/tr10/#Step_3
1782-
*
1783-
* The collation algorithm standard goes on to state:
1784-
*
1785-
* "By default, the algorithm makes use of three fully-customizable levels.
1786-
* For the Latin script, these levels correspond roughly to:
1787-
*
1788-
* alphabetic ordering
1789-
*
1790-
* diacritic ordering
1791-
*
1792-
* case ordering.
1755+
* If possible, set ssup->comparator to a function which can be used to
1756+
* directly compare two datums. If we can do this, we'll avoid the
1757+
* overhead of a trip through the fmgr layer for every comparison,
1758+
* which can be substantial.
17931759
*
1794-
* A final level may be used for tie-breaking between strings not otherwise
1795-
* distinguished."
1760+
* Most typically, we'll set the comparator to bttextfastcmp_locale,
1761+
* which uses strcoll() to perform comparisons. However, if LC_COLLATE
1762+
* = C, we can make things quite a bit faster with bttextfastcmp_c,
1763+
* which uses memcmp() rather than strcoll().
17961764
*
1797-
* It is generally expected that most non-equal keys will have their
1798-
* comparisons resolved at the primary level. If enough comparisons can be
1799-
* resolved with just 4 or 8 byte abbreviated keys, this optimization is
1800-
* very effective (although if there are many tie-breakers that largely
1801-
* only perform cheap memcmp() calls, that is also much faster than the
1802-
* unoptimized case - see bttext_abbrev_abort()).
1803-
*
1804-
* We may need a collation-sensitive comparison. To make things faster,
1805-
* we'll figure out the collation based on the locale id and cache the
1806-
* result. Also, since strxfrm()/strcoll() require NUL-terminated inputs,
1807-
* prepare one or two palloc'd buffers to use as temporary workspace. In
1808-
* the ad-hoc comparison case we only use palloc'd buffers when we need
1809-
* more space than we're comfortable allocating on the stack, but here we
1810-
* can keep the buffers around for the whole sort, so it makes sense to
1811-
* allocate them once and use them unconditionally.
1765+
* There is a further exception on Windows. When the database encoding
1766+
* is UTF-8 and we are not using the C collation, complex hacks are
1767+
* required. We don't currently have a comparator that handles that case,
1768+
* so we fall back on the slow method of having the sort code invoke
1769+
* bttextcmp() via the fmgr trampoline.
18121770
*/
1813-
tss = palloc(sizeof(TextSortSupport));
1814-
#ifdef HAVE_LOCALE_T
1815-
tss->locale = 0;
1771+
if (lc_collate_is_c(collid))
1772+
ssup->comparator = bttextfastcmp_c;
1773+
#ifdef WIN32
1774+
else if (GetDatabaseEncoding() == PG_UTF8)
1775+
return;
18161776
#endif
1817-
1818-
if (collid != DEFAULT_COLLATION_OID)
1777+
else
18191778
{
1820-
if (!OidIsValid(collid))
1779+
ssup->comparator = bttextfastcmp_locale;
1780+
locale_aware = true;
1781+
1782+
/*
1783+
* We need a collation-sensitive comparison. To make things faster,
1784+
* we'll figure out the collation based on the locale id and cache the
1785+
* result.
1786+
*/
1787+
if (collid != DEFAULT_COLLATION_OID)
18211788
{
1822-
/*
1823-
* This typically means that the parser could not resolve a
1824-
* conflict of implicit collations, so report it that way.
1825-
*/
1826-
ereport(ERROR,
1827-
(errcode(ERRCODE_INDETERMINATE_COLLATION),
1828-
errmsg("could not determine which collation to use for string comparison"),
1829-
errhint("Use the COLLATE clause to set the collation explicitly.")));
1830-
}
1789+
if (!OidIsValid(collid))
1790+
{
1791+
/*
1792+
* This typically means that the parser could not resolve a
1793+
* conflict of implicit collations, so report it that way.
1794+
*/
1795+
ereport(ERROR,
1796+
(errcode(ERRCODE_INDETERMINATE_COLLATION),
1797+
errmsg("could not determine which collation to use for string comparison"),
1798+
errhint("Use the COLLATE clause to set the collation explicitly.")));
1799+
}
18311800
#ifdef HAVE_LOCALE_T
1832-
tss->locale = pg_newlocale_from_collation(collid);
1801+
tss->locale = pg_newlocale_from_collation(collid);
18331802
#endif
1803+
}
18341804
}
18351805

18361806
/*
1837-
* If LC_COLLATE = C, we can make things quite a bit faster by using
1838-
* memcmp() rather than strcoll(). To minimize the per-comparison
1839-
* overhead, we make this decision just once for the whole sort.
1807+
* It's possible that there are platforms where the use of abbreviated
1808+
* keys should be disabled at compile time. Having only 4 byte datums
1809+
* could make worst-case performance drastically more likely, for example.
1810+
* Moreover, Darwin's strxfrm() implementations is known to not effectively
1811+
* concentrate a significant amount of entropy from the original string in
1812+
* earlier transformed blobs. It's possible that other supported platforms
1813+
* are similarly encumbered. However, even in those cases, the abbreviated
1814+
* keys optimization may win, and if it doesn't, the "abort abbreviation"
1815+
* code may rescue us. So, for now, we don't disable this anywhere on the
1816+
* basis of performance.
18401817
*
1841-
* There is no reason to not at least perform fmgr elision on builds where
1842-
* abbreviation is disabled.
1818+
* On Windows, however, strxfrm() seems to be unreliable on some machines,
1819+
* so we categorically disable this optimization there.
18431820
*/
1844-
if (lc_collate_is_c(collid))
1845-
ssup->abbrev_full_comparator = ssup->comparator = bttextfastcmp_c;
1846-
else
1847-
ssup->abbrev_full_comparator = ssup->comparator = bttextfastcmp_locale;
1821+
#ifdef WIN32
1822+
abbreviate = false;
1823+
#endif
18481824

1849-
if (!lc_collate_is_c(collid) || abbreviate)
1825+
/*
1826+
* If we're using abbreviated keys, or if we're using a locale-aware
1827+
* comparison, we need to initialize a TextSortSupport object. Both cases
1828+
* will make use of the temporary buffers we initialize here for scratch
1829+
* space, and the abbreviation case requires additional state.
1830+
*/
1831+
if (abbreviate || locale_aware)
18501832
{
1851-
/*
1852-
* Abbreviated case requires temp buffers for strxfrm() copying.
1853-
* bttextfastcmp_locale() also uses these buffers (even if abbreviation
1854-
* isn't used), while bttextfast_c() does not.
1855-
*/
1833+
tss = palloc(sizeof(TextSortSupport));
18561834
tss->buf1 = palloc(TEXTBUFLEN);
18571835
tss->buflen1 = TEXTBUFLEN;
18581836
tss->buf2 = palloc(TEXTBUFLEN);
18591837
tss->buflen2 = TEXTBUFLEN;
1838+
#ifdef HAVE_LOCALE_T
1839+
tss->locale = locale;
1840+
#endif
18601841
ssup->ssup_extra = tss;
1861-
}
1862-
1863-
if (!abbreviate)
1864-
return;
18651842

1866-
initHyperLogLog(&tss->abbr_card, 10);
1867-
initHyperLogLog(&tss->full_card, 10);
1868-
1869-
/*
1870-
* Change comparator to be abbreviation-based -- abbreviated version will
1871-
* probably ultimately be used during sorting proper, but core code may
1872-
* switch back to authoritative comparator should abbreviation be aborted
1873-
*/
1874-
ssup->comparator = bttextcmp_abbrev;
1875-
ssup->abbrev_converter = bttext_abbrev_convert;
1876-
ssup->abbrev_abort = bttext_abbrev_abort;
1843+
/*
1844+
* If possible, plan to use the abbreviated keys optimization. The
1845+
* core code may switch back to authoritative comparator should
1846+
* abbreviation be aborted.
1847+
*/
1848+
if (abbreviate)
1849+
{
1850+
initHyperLogLog(&tss->abbr_card, 10);
1851+
initHyperLogLog(&tss->full_card, 10);
1852+
ssup->abbrev_full_comparator = ssup->comparator;
1853+
ssup->comparator = bttextcmp_abbrev;
1854+
ssup->abbrev_converter = bttext_abbrev_convert;
1855+
ssup->abbrev_abort = bttext_abbrev_abort;
1856+
}
1857+
}
18771858
}
18781859

18791860
/*

0 commit comments

Comments
 (0)
0