8000 Merge branch 'master' into speedup_type_creation · python/cpython@9026dcc · GitHub
[go: up one dir, main page]
Skip to content

Commit 9026dcc

Browse files
pitroupitrou
committed
Merge branch 'master' into speedup_type_creation
2 parents c39d280 + 78e24d4 commit 9026dcc

File tree

14 files changed

+105
-60
lines changed

14 files changed

+105
-60
lines changed

‎.github/CODEOWNERS

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,13 @@
1414
**/*hashlib* @python/crypto-team
1515
**/*pyhash* @python/crypto-team
1616

17-
# Import (including importlib)
18-
**/*import* @python/import-team
17+
# Import (including importlib).
18+
# Ignoring importlib.h so as to not get flagged on
19+
# all pull requests that change the the emitted
20+
# bytecode.
21+
**/*import*.c @python/import-team
22+
**/*import*.py @python/import-team
23+
1924

2025
# SSL
2126
**/*ssl* @python/crypto-team

‎Doc/howto/regex.rst

Lines changed: 2 additions & 2 deletions
< F438 div data-testid="deletion diffstat" class="DiffSquares-module__diffSquare--h5kjy DiffSquares-module__deletion--hKV3q">
Original file line numberDiff line numberDiff line change
@@ -1140,12 +1140,12 @@ new string value and the number of replacements that were performed::
11401140
>>> p.subn('colour', 'no colours at all')
11411141
('no colours at all', 0)
11421142

1143-
Empty matches are replaced only when they're not adjacent to a previous match.
1143+
Empty matches are replaced only when they're not adjacent to a previous empty match.
11441144
::
11451145

11461146
>>> p = re.compile('x*')
11471147
>>> p.sub('-', 'abxd')
1148-
'-a-b-d-'
1148+
'-a-b--d-'
11491149

11501150
If *replacement* is a string, any backslash escapes in it are processed. That
11511151
is, ``\n`` is converted to a single newline character, ``\r`` is converted to a

‎Doc/library/re.rst

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -708,12 +708,15 @@ form.
708708
That way, separator components are always found at the same relative
709709
indices within the result list.
710710

711-
The pattern can match empty strings. ::
711+
Empty matches for the pattern split the string only when not adjacent
712+
to a previous empty match.
712713

713714
>>> re.split(r'\b', 'Words, words, words.')
714715
['', 'Words', ', ', 'words', ', ', 'words', '.']
716+
>>> re.split(r'\W*', '...words...')
717+
['', '', 'w', 'o', 'r', 'd', 's', '', '']
715718
>>> re.split(r'(\W*)', '...words...')
716-
['', '...', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '']
719+
['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', '']
717720

718721
.. versionchanged:: 3.1
719722
Added the optional flags argument.
@@ -778,8 +781,8 @@ form.
778781
The optional argument *count* is the maximum number of pattern occurrences to be
779782
replaced; *count* must be a non-negative integer. If omitted or zero, all
780783
occurrences will be replaced. Empty matches for the pattern are replaced only
781-
when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
782-
``'-a-b-c-'``.
784+
when not adjacent to a previous empty match, so ``sub('x*', '-', 'abxd')`` returns
785+
``'-a-b--d-'``.
783786

784787
In string-type *repl* arguments, in addition to the character escapes and
785788
backreferences described above,
@@ -805,6 +808,9 @@ form.
805808
Unknown escapes in *repl* consisting of ``'\'`` and an ASCII letter
806809
now are errors.
807810

811+
Empty matches for the pattern are replaced when adjacent to a previous
812+
non-empty match.
813+
808814

809815
.. function:: subn(pattern, repl, string, count=0, flags=0)
810816

‎Doc/whatsnew/3.7.rst

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -881,8 +881,9 @@ Changes in the Python API
881881
* The result of splitting a string on a :mod:`regular expression <re>`
882882
that could match an empty string has been changed. For example
883883
splitting on ``r'\s*'`` will now split not only on whitespaces as it
884-
did previously, but also between any pair of non-whitespace
885-
characters. The previous behavior can be restored by changing the pattern
884+
did previously, but also on empty strings before all non-whitespace
885+
characters and just before the end of the string.
886+
The previous behavior can be restored by changing the pattern
886887
to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since
887888
Python 3.5.
888889

@@ -893,7 +894,13 @@ Changes in the Python API
893894
positions 2--3. To match only blank lines, the pattern should be rewritten
894895
as ``r'(?m)^[^\S\n]*$'``.
895896

896-
(Contributed by Serhiy Storchaka in :issue:`25054`.)
897+
:func:`re.sub()` now replaces empty matches adjacent to a previous
898+
non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now
899+
``'-a-b--d-'`` instead of ``'-a-b--d-'`` (the first minus between 'b' and
900 10000 +
'd' replaces 'x', and the second minus replaces an empty string between
901+
'x' and 'd').
902+
903+
(Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.)
897904

898905
* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most
899906
recent to be more consistent with :mod:`traceback`.

‎Lib/ast.py

Lines changed: 21 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,6 @@ def parse(source, filename='<unknown>', mode='exec'):
3535
return compile(source, filename, mode, PyCF_ONLY_AST)
3636

3737

38-
_NUM_TYPES = (int, float, complex)
39-
4038
def literal_eval(node_or_string):
4139
"""
4240
Safely evaluate an expression node or a string containing a Python
@@ -48,6 +46,21 @@ def literal_eval(node_or_string):
4846
node_or_string = parse(node_or_string, mode='eval')
4947
if isinstance(node_or_string, Expression):
5048
node_or_string = node_or_string.body
49+
def _convert_num(node):
50+
if isinstance(node, Constant):
51+
if isinstance(node.value, (int, float, complex)):
52+
return node.value
53+
elif isinstance(node, Num):
54+
return node.n
55+
raise ValueError('malformed node or string: ' + repr(node))
56+
def _convert_signed_num(node):
57+
if isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)):
58+
operand = _convert_num(node.operand)
59+
if isinstance(node.op, UAdd):
60+
return + operand
61+
else:
62+
return - operand
63+
return _convert_num(node)
5164
def _convert(node):
5265
if isinstance(node, Constant):
5366
return node.value
@@ -62,26 +75,19 @@ def _convert(node):
6275
elif isinstance(node, Set):
6376
return set(map(_convert, node.elts))
6477
elif isinstance(node, Dict):
65-
return dict((_convert(k), _convert(v)) for k, v
66-
in zip(node.keys, node.values))
78+
return dict(zip(map(_convert, node.keys),
79+
map(_convert, node.values)))
6780
elif isinstance(node, NameConstant):
6881
return node.value
69-
elif isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)):
70-
operand = _convert(node.operand)
71-
if isinstance(operand, _NUM_TYPES):
72-
if isinstance(node.op, UAdd):
73-
return + operand
74-
else:
75-
return - operand
7682
elif isinstance(node, BinOp) and isinstance(node.op, (Add, Sub)):
77-
left = _convert(node.left)
78-
right = _convert(node.right)
79-
if isinstance(left, _NUM_TYPES) and isinstance(right, _NUM_TYPES):
83+
left = _convert_signed_num(node.left)
84+
right = _convert_num(node.right)
85+
if isinstance(left, (int, float)) and isinstance(right, complex):
8086
if isinstance(node.op, Add):
8187
return left + right
8288
else:
8389
return left - right
84-
raise ValueError('malformed node or string: ' + repr(node))
90+
return _convert_signed_num(node)
8591
return _convert(node_or_string)
8692

8793

‎Lib/test/test_ast.py

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -551,14 +551,37 @@ def test_literal_eval(self):
551551
self.assertEqual(ast.literal_eval('{1, 2, 3}'), {1, 2, 3})
552552
self.assertEqual(ast.literal_eval('b"hi"'), b"hi")
553553
self.assertRaises(ValueError, ast.literal_eval, 'foo()')
554+
self.assertEqual(ast.literal_eval('6'), 6)
555+
self.assertEqual(ast.literal_eval('+6'), 6)
554556
self.assertEqual(ast.literal_eval('-6'), -6)
555-
self.assertEqual(ast.literal_eval('-6j+3'), 3-6j)
556557
self.assertEqual(ast.literal_eval('3.25'), 3.25)
557-
558-
def test_literal_eval_issue4907(self):
559-
self.assertEqual(ast.literal_eval('2j'), 2j)
560-
self.assertEqual(ast.literal_eval('10 + 2j'), 10 + 2j)
561-
self.assertEqual(ast.literal_eval('1.5 - 2j'), 1.5 - 2j)
558+
self.assertEqual(ast.literal_eval('+3.25'), 3.25)
559+
self.assertEqual(ast.literal_eval('-3.25'), -3.25)
560+
self.assertEqual(repr(ast.literal_eval('-0.0')), '-0.0')
561+
self.assertRaises(ValueError, ast.literal_eval, '++6')
562+
self.assertRaises(ValueError, ast.literal_eval, '+True')
563+
self.assertRaises(ValueError, ast.literal_eval, '2+3')
564+
565+
def test_literal_eval_complex(self):
566+
# Issue #4907
567+
self.assertEqual(ast.literal_eval('6j'), 6j)
568+
self.assertEqual(ast.literal_eval('-6j'), -6j)
569+
self.assertEqual(ast.literal_eval('6.75j'), 6.75j)
570+
self.assertEqual(ast.literal_eval('-6.75j'), -6.75j)
571+
self.assertEqual(ast.literal_eval('3+6j'), 3+6j)
572+
self.assertEqual(ast.literal_eval('-3+6j'), -3+6j)
573+
self.assertEqual(ast.literal_eval('3-6j'), 3-6j)
574+
self.assertEqual(ast.literal_eval('-3-6j'), -3-6j)
575+
self.assertEqual(ast.literal_eval('3.25+6.75j'), 3.25+6.75j)
576+
self.assertEqual(ast.literal_eval('-3.25+6.75j'), -3.25+6.75j)
577+
self.assertEqual(ast.literal_eval('3.25-6.75j'), 3.25-6.75j)
578+
self.assertEqual(ast.literal_eval('-3.25-6.75j'), -3.25-6.75j)
579+
self.assertEqual(ast.literal_eval('(3+6j)'), 3+6j)
580+
self.assertRaises(ValueError, ast.literal_eval, '-6j+3')
581+
self.assertRaises(ValueError, ast.literal_eval, '-6j+3j')
582+
self.assertRaises(ValueError, ast.literal_eval, '3+-6j')
583+
self.assertRaises(ValueError, ast.literal_eval, '3+(0+6j)')
584+
self.assertRaises(ValueError, ast.literal_eval, '-(3+6j)')
562585

563586
def test_bad_integer(self):
564587
# issue13436: Bad error message with invalid numeric values
@@ -1077,11 +1100,11 @@ def test_literal_eval(self):
10771100
ast.copy_location(new_left, binop.left)
10781101
binop.left = new_left
10791102

1080-
new_right = ast.Constant(value=20)
1103+
new_right = ast.Constant(value=20j)
10811104
ast.copy_location(new_right, binop.right)
10821105
binop.right = new_right
10831106

1084-
self.assertEqual(ast.literal_eval(binop), 30)
1107+
self.assertEqual(ast.literal_eval(binop), 10+20j)
10851108

10861109

10871110
def main():

‎Lib/test/test_grammar.py

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -575,6 +575,10 @@ def f(*args, **kwargs):
575575
self.assertEqual(f(spam='fried', **{'eggs':'scrambled'}),
576576
((), {'eggs':'scrambled', 'spam':'fried'}))
577577

578+
# Check ast errors in *args and *kwargs
579+
check_syntax_error(self, "f(*g(1=2))")
580+
check_syntax_error(self, "f(**g(1=2))")
581+
578582
# argument annotation tests
579583
def f(x) -> list: pass
580584
self.assertEqual(f.__annotations__, {'return': list})
@@ -616,10 +620,6 @@ def f(x=1): return closure
616620
def f(*, k=1): return closure
617621
def f() -> int: return closure
618622

619-
# Check ast errors in *args and *kwargs
620-
check_syntax_error(self, "f(*g(1=2))")
621-
check_syntax_error(self, "f(**g(1=2))")
622-
623623
# Check trailing commas are permitted in funcdef argument list
624624
def f(a,): pass
625625
def f(*args,): pass
@@ -1091,7 +1091,6 @@ def test_try(self):
10911091
try: 1/0
10921092
except EOFError: pass
10931093
except TypeError as msg: pass
1094-
except RuntimeError as msg: pass
10951094
except: pass
10961095
else: pass
10971096
try: 1/0
@@ -1200,7 +1199,7 @@ def test_selectors(self):
12001199
d[1,2] = 3
12011200
d[1,2,3] = 4
12021201
L = list(d)
1203-
L.sort(key=lambda x: x if isinstance(x, tuple) else ())
1202+
L.sort(key=lambda x: (type(x).__name__, x))
12041203
self.assertEqual(str(L), '[1, (1,), (1, 2), (1, 2, 3)]')
12051204

12061205
def test_atoms(self):

‎Lib/test/test_inspect.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2074,7 +2074,7 @@ def p(name): return signature.parameters[name].default
20742074
self.assertEqual(p('f'), False)
20752075
self.assertEqual(p('local'), 3)
20762076
self.assertEqual(p('sys'), sys.maxsize)
2077-
self.assertEqual(p('exp'), sys.maxsize - 1)
2077+
self.assertNotIn('exp', signature.parameters)
20782078

20792079
test_callable(object)
20802080

‎Lib/test/test_re.py

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -213,11 +213,6 @@ def test_bug_114660(self):
213213
self.assertEqual(re.sub(r'(\S)\s+(\S)', r'\1 \2', 'hello there'),
214214
'hello there')
215215

216-
def test_bug_462270(self):
217-
# Test for empty sub() behaviour, see SF bug #462270
218-
self.assertEqual(re.sub('x*', '-', 'abxd'), '-a-b-d-')
219-
self.assertEqual(re.sub('x+', '-', 'abxd'), 'ab-d')
220-
221216
def test_symbolic_groups(self):
222217
re.compile(r'(?P<a>x)(?P=a)(?(a)y)')
223218
re.compile(r'(?P<a1>x)(?P=a1)(?(a1)y)')
@@ -331,10 +326,10 @@ def test_re_split(self):
331326
['', 'a', '', '', 'c'])
332327

333328
for sep, expected in [
334-
(':*', ['', 'a', 'b', 'c', '']),
335-
('(?::*)', ['', 'a', 'b', 'c', '']),
336-
('(:*)', ['', ':', 'a', ':', 'b', '::', 'c', '', '']),
337-
('(:)*', ['', ':', 'a', ':', 'b', ':', 'c', None, '']),
329+
(':*', ['', '', 'a', '', 'b', '', 'c', '']),
330+
('(?::*)', ['', '', 'a', '', 'b', '', 'c', '']),
331+
('(:*)', ['', ':', '', '', 'a', ':', '', '', 'b', '::', '', '', 'c', '', '']),
332+
('(:)*', ['', ':', '', None, 'a', ':', '', None, 'b', ':', '', None, 'c', None, '']),
338333
]:
339334
with self.subTest(sep=sep):
340335
self.assertTypedEqual(re.split(sep, ':a:b::c'), expected)
@@ -357,7 +352,7 @@ def test_qualified_re_split(self):
357352
self.assertEqual(re.split("(:+)", ":a:b::c", maxsplit=2),
358353
['', ':', 'a', ':', 'b::c'])
359354
self.assertEqual(re.split("(:*)", ":a:b::c", maxsplit=2),
360-
['', ':', 'a', ':', 'b::c'])
355+
['', ':', '', '', 'a:b::c'])
361356

362357
def test_re_findall(self):
363358
self.assertEqual(re.findall(":+", "abc"), [])
@@ -1753,13 +1748,13 @@ def test_match_repr(self):
17531748
def test_zerowidth(self):
17541749
# Issues 852532, 1647489, 3262, 25054.
17551750
self.assertEqual(re.split(r"\b", "a::bc"), ['', 'a', '::', 'bc', ''])
1756-
self.assertEqual(re.split(r"\b|:+", "a::bc"), ['', 'a', '', 'bc', ''])
1757-
self.assertEqual(re.split(r"(?<!\w)(?=\w)|:+", "a::bc"), ['', 'a', 'bc'])
1751+
self.assertEqual(re.split(r"\b|:+", "a::bc"), ['', 'a', '', '', 'bc', ''])
1752+
self.assertEqual(re.split(r"(?<!\w)(?=\w)|:+", "a::bc"), ['', 'a', '', 'bc'])
17581753
self.assertEqual(re.split(r"(?<=\w)(?!\w)|:+", "a::bc"), ['a', '', 'bc', ''])
17591754

17601755
self.assertEqual(re.sub(r"\b", "-", "a::bc"), '-a-::-bc-')
1761-
self.assertEqual(re.sub(r"\b|:+", "-", "a::bc"), '-a--bc-')
1762-
self.assertEqual(re.sub(r"(\b|:+)", r"[\1]", "a::bc"), '[]a[][::]bc[]')
1756+
self.assertEqual(re.sub(r"\b|:+", "-", "a::bc"), '-a---bc-')
1757+
self.assertEqual(re.sub(r"(\b|:+)", r"[\1]", "a::bc"), '[]a[][::][]bc[]')
17631758

17641759
self.assertEqual(re.findall(r"\b|:+", "a::bc"), ['', '', '::', '', ''])
17651760
self.assertEqual(re.findall(r"\b|\w+", "a::bc"),

‎Lib/test/test_syntax.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -668,12 +668,12 @@ def test_kwargs_last(self):
668668
"positional argument follows keyword argument")
669669

670670
def test_kwargs_last2(self):
671-
self._check_error("int(**{base: 10}, '2')",
671+
self._check_error("int(**{'base': 10}, '2')",
672672
"positional argument follows "
673673
"keyword argument unpacking")
674674

675675
def test_kwargs_last3(self):
676-
self._check_error("int(**{base: 10}, *['2'])",
676+
self._check_error("int(**{'base': 10}, *['2'])",
677677
"iterable argument unpacking follows "
678678
"keyword argument unpacking")
679679

‎Misc/NEWS.d/next/Library/2017-10-18-17-29-30.bpo-31778.B6vAkP.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
ast.literal_eval() is now more strict. Addition and subtraction of
2+
arbitrary numbers no longer allowed.

‎Misc/NEWS.d/next/Library/2017-12-13-20-31-30.bpo-32308.CUbsb2.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
:func:`re.sub()` now replaces empty matches adjacent to a previous non-empty
2+
match.

‎Modules/_sre.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -955,7 +955,7 @@ _sre_SRE_Pattern_split_impl(PatternObject *self, PyObject *string,
955955
}
956956

957957
n = n + 1;
958-
state.must_advance = 1;
958+
state.must_advance = (state.ptr == state.start);
959959
last = state.start = state.ptr;
960960

961961
}
@@ -1109,7 +1109,7 @@ pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string,
11091109

11101110
i = e;
11111111
n = n + 1;
1112-
state.must_advance = 1;
1112+
state.must_advance = (state.ptr == state.start);
11131113
state.start = state.ptr;
11141114
}
11151115

‎Objects/typeobject.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2375,7 +2375,7 @@ type_new(PyTypeObject *metatype, PyObject *args, PyObject *kwds)
23752375
}
23762376

23772377
/* SF bug 475327 -- if that didn't trigger, we need 3
2378-
arguments. but PyArg_ParseTupleAndKeywords below may give
2378+
arguments. but PyArg_ParseTuple below may give
23792379
a msg saying type() needs exactly 3. */
23802380
if (nargs != 3) {
23812381
PyErr_SetString(PyExc_TypeError,

0 commit comments

Comments
 (0)
0