10000 gh-91524: Speed up the regular expression substitution by serhiy-storchaka · Pull Request #91525 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-91524: Speed up the regular expression substitution #91525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Oct 23, 2022
Merged
Prev Previous commit
Next Next commit
Merge branch 'main' into re-compile-template
  • Loading branch information
gpshead committed Oct 23, 2022
commit 2271aa89a429358d540c0a4152a064ff9ffc4759
19 changes: 15 additions & 4 deletions Doc/whatsnew/3.11.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1204,11 +1204,22 @@ This section covers specific optimizations independent of the
with speeds over 100 times higher for a ≈60 MiB file.
(Contributed by msoxzw in :gh:`91487`.)

* Speed up the regular expression substitution (functions :func:`re.sub` and
:func:`re.subn` and corresponding :class:`re.Pattern` methods) for
replacement strings containing group references by 2--3 times.
(Contributed by Serhiy Storchaka in :issue:`91524`.)
* :mod:`math` functions :func:`~math.comb` and :func:`~math.perm` are now
≈10 times faster for large arguments (with a larger speedup for larger *k*).
(Contributed by Serhiy Storchaka in :issue:`37295`.)

* The :mod:`statistics` functions :func:`~statistics.mean`,
:func:`~statistics.variance` and :func:`~statistics.stdev` now consume
iterators in one pass rather than converting them to a :class:`list` first.
This is twice as fast and can save substantial memory.
(Contributed by Raymond Hettinger in :gh:`90415`.)

* :func:`unicodedata.normalize`
now normalizes pure-ASCII strings in constant time.
(Contributed by Dong-hee Na in :issue:`44987`.)


.. _whatsnew311-faster-cpython:

Faster CPython
==============
Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,11 @@ Optimizations
process, which improves performance by 1-5%.
(Contributed by Kevin Modzelewski in :gh:`90536`.)

* Speed up the regular expression substitution (functions :func:`re.sub` and
:func:`re.subn` and corresponding :class:`re.Pattern` methods) for
replacement strings containing group references by 2--3 times.
(Contributed by Serhiy Storchaka in :issue:`91524`.)


CPython bytecode changes
========================
Expand Down
1 change: 1 addition & 0 deletions Lib/re/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ def compile(pattern, flags=0):
def purge():
"Clear the regular expression caches"
_cache.clear()
_cache2.clear()
_compile_template.cache_clear()

def template(pattern, flags=0):
Expand Down
2 changes: 1 addition & 1 deletion Lib/re/_constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

# update when constants are added or removed

MAGIC = 20220409
MAGIC = 20221023

from _sre import MAXREPEAT, MAXGROUPS

Expand Down
4 changes: 2 additions & 2 deletions Modules/_sre/clinic/sre.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 2 additions & 4 deletions Modules/_sre/sre.c
Original file line number Diff line number Diff line change
Expand Up @@ -760,11 +760,9 @@ import(const char* module, const char* function)
name = PyUnicode_FromString(module);
if (!name)
return NULL;
mod = PyImport_Import(name);
Py_DECREF(name);
if (!mod)
func = _PyImport_GetModuleAttrString(module, function);
if (!func)
return NULL;
func = PyObject_GetAttrString(mod, function);
Py_DECREF(mod);
return func;
}
Expand Down
2 changes: 1 addition & 1 deletion Modules/_sre/sre_constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
* See the sre.c file for information on usage and redistribution.
*/

#define SRE_MAGIC 20220409
#define SRE_MAGIC 20221023
#define SRE_OP_FAILURE 0
#define SRE_OP_SUCCESS 1
#define SRE_OP_ANY 2
Expand Down
You are viewing a condensed version of this merge commit. You can view the full changes here.
0