From 713486144e529b23d3de72b76f51267db64ea5b2 Mon Sep 17 00:00:00 2001 From: Batuhan Taskaya Date: Fri, 15 Nov 2019 18:13:58 +0300 Subject: [PATCH 1/5] bpo-3530: advice fix_missing_locations usage on new node introduction with NodeTransformer --- Doc/library/ast.rst | 7 ++++++- .../Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index b468f4235df300..2b375b0263b41e 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -300,7 +300,7 @@ and classes for traversing abstract syntax trees: class RewriteName(NodeTransformer): def visit_Name(self, node): - return copy_location(Subscript( + return Subscript( value=Name(id='data', ctx=Load()), slice=Index(value=Constant(value=node.id)), ctx=node.ctx @@ -314,6 +314,11 @@ and classes for traversing abstract syntax trees: statement nodes), the visitor may also return a list of nodes rather than just a single node. + If :class:`NodeTransformer` introduces new nodes (that weren't part of + original tree) without giving them location information (such as + :attr:`lineno`), :func:`fix_missing_locations` should be called with + new tree to apply location information for compiler. + Usually you use the transformer like this:: node = YourTransformer().visit(node) diff --git a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst new file mode 100644 index 00000000000000..5a357ee801d8c3 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst @@ -0,0 +1,2 @@ +In ast doc, fix misleading NodeTransformer example and add +fix_missing_locations advice. From 0dca9b3041418f76c2de294f8939b16a0128cc0c Mon Sep 17 00:00:00 2001 From: isidentical Date: Sun, 12 Jan 2020 22:54:32 +0300 Subject: [PATCH 2/5] fix some markup and typos --- Doc/library/ast.rst | 2 +- .../Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 2b375b0263b41e..ad1e1c5736f0af 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -317,7 +317,7 @@ and classes for traversing abstract syntax trees: If :class:`NodeTransformer` introduces new nodes (that weren't part of original tree) without giving them location information (such as :attr:`lineno`), :func:`fix_missing_locations` should be called with - new tree to apply location information for compiler. + the new sub-tree to recalculate the location information. Usually you use the transformer like this:: diff --git a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst index 5a357ee801d8c3..0d8908eacc3b5f 100644 --- a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst +++ b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst @@ -1,2 +1,2 @@ -In ast doc, fix misleading NodeTransformer example and add -fix_missing_locations advice. +In :mod:`ast` doc, fix misleading ``NodeTransformer`` example and add +``fix_missing_locations`` advice. From 2b827c8e3bbed236ff68efa501f11666fee95918 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Batuhan=20Ta=C5=9Fkaya?= <47358913+isidentical@users.noreply.github.com> Date: Sun, 12 Jan 2020 23:02:29 +0300 Subject: [PATCH 3/5] Apply suggestions from code review Co-Authored-By: Pablo Galindo --- .../Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst index 0d8908eacc3b5f..597d5f7293115d 100644 --- a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst +++ b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst @@ -1,2 +1,2 @@ -In :mod:`ast` doc, fix misleading ``NodeTransformer`` example and add -``fix_missing_locations`` advice. +in the :mod:`ast` module documentation, fix a misleading ``NodeTransformer`` example and add +advice on when to use the ``fix_missing_locations`` function. From 2fd17e602be973c8e18787dbe13f16678884b820 Mon Sep 17 00:00:00 2001 From: isidentical Date: Sun, 12 Jan 2020 23:15:51 +0300 Subject: [PATCH 4/5] add an example --- Doc/library/ast.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index ad1e1c5736f0af..0a4e7c6207e717 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -317,7 +317,10 @@ and classes for traversing abstract syntax trees: If :class:`NodeTransformer` introduces new nodes (that weren't part of original tree) without giving them location information (such as :attr:`lineno`), :func:`fix_missing_locations` should be called with - the new sub-tree to recalculate the location information. + the new sub-tree to recalculate the location information:: + + tree = ast.parse('foo', mode='eval') + new_tree = fix_missing_locations(RewriteName().visit(tree)) Usually you use the transformer like this:: From f4124716cc2dedd92b659bd9046d257a957f0c99 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Batuhan=20Ta=C5=9Fkaya?= <47358913+isidentical@users.noreply.github.com> Date: Sun, 12 Jan 2020 23:28:41 +0300 Subject: [PATCH 5/5] Update Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst Co-Authored-By: Pablo Galindo --- .../next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst index 597d5f7293115d..65f1a6d156a120 100644 --- a/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst +++ b/Misc/NEWS.d/next/Documentation/2019-11-17-11-53-10.bpo-3530.8zFUMc.rst @@ -1,2 +1,2 @@ -in the :mod:`ast` module documentation, fix a misleading ``NodeTransformer`` example and add +In the :mod:`ast` module documentation, fix a misleading ``NodeTransformer`` example and add advice on when to use the ``fix_missing_locations`` function.