8000 gh-90117: handle dict and mapping views in pprint by devdanzin · Pull Request #30135 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-90117: handle dict and mapping views in pprint #30135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
May 20, 2025

Conversation

devdanzin
Copy link
Contributor
@devdanzin devdanzin commented Dec 16, 2021

This PR adds two methods to PrettyPrinter: one to handle dict_keys and dict_values (sorted with _safe_key), another to handle dict_items (sorted using _safe_tuple).

Design and implementation open for discussion, thanks in advance.

https://bugs.python.org/issue45959

@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@devdanzin

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@AlexWaygood AlexWaygood added the type-feature A feature request or enhancement label Dec 17, 2021
@devdanzin
Copy link
Contributor Author

Thank you for the review, Alex!

@AlexWaygood
Copy link
Member

Thank you for the review, Alex!

Pleasure :) You can add a News entry to your PR using https://blurb-it.herokuapp.com

@devdanzin
Copy link
Contributor Author

Thank you for reviewing, Éric!

@github-actions
Copy link

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Stale PR or inactive for long period of time. label Jan 18, 2022
Copy link
Contributor
@BvB93 BvB93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also the collections.abc.KeysView, ValuesView and ItemsView classes, all of which should be able to reuse the logic used here for the various dict views. For the sake of consistency, I think it would be worthwhile to include them as well.

@merwok
Copy link
Member
merwok commented Jan 26, 2022

The sake of consistency is not a sufficient reason to do things. The question should be: what is the feature added by handling dict views ABCs in pprint? Do people pretty-print these ABCs? Would registering them automatically handle instance of concrete subclasses of the ABCs? In that case, should that replace the code added in this PR?

@BvB93
Copy link
Contributor
BvB93 commented Jan 26, 2022

The question should be: what is the feature added by handling dict views ABCs in pprint? Do people pretty-print these ABCs?

Data point of 1, but I've personally run into this exact issue before, both with normal dict views as well as custom mapping views derived from the collections.abc ABCs.

Would registering them automatically handle instance of concrete subclasses of the ABCs? In that case, should that replace the code added in this PR?

Registering the collections.abc.MappingView baseclass (which is the one that actually implements __repr__ for the mapping views) should do most of the work.

The only caveat is that because the mapping views share identical __repr__ methods by default it's not possible to distinguish between, e.g., ItemsView and KeysView simply by using the __repr__ method as a dictionary key. Nevertheless, some minor changes in the logic of _pprint_dict_view should be able to deal with this.

@@ -233,8 +233,11 @@ def _pprint_ordered_dict(self, object, stream, indent, allowance, context, level
-    def _pprint_dict_view(self, object, stream, indent, allowance, context, level, items=False):
-        key = _safe_tuple if items else _safe_key
+    def _pprint_dict_view(self, object, stream, indent, allowance, context, level):
+        if isinstance(object, (self._dict_items_view, _collections.abc.ItemsView)):
+            key = _safe_tuple
+        else:
+            key = _safe_key
@@ -255,11 +258,10 @@ def _pprint_dict_view(self, object, stream, indent, allowance, context, level, i
-    def _pprint_dict_items_view(self, object, stream, indent, allowance, context, level):
-        self._pprint_dict_view(object, stream, indent, allowance, context, level, items=True)
-
     _dict_items_view = type({}.items())
-    _dispatch[_dict_items_view.__repr__] = _pprint_dict_items_view
+    _dispatch[_dict_items_view.__repr__] = _pprint_dict_view
+
+    _dispatch[_collections.abc.MappingView.__repr__] = _pprint_dict_view

@github-actions github-actions bot removed the stale Stale PR or inactive for long period of time. label Jan 27, 2022
@devdanzin
Copy link
Contributor Author

I believe this PR is now ready for review. Any feedback is most welcome!

It tries to keep the changes self-contained and use the same code style as the rest of the module, matching the code for pretty printing of dicts as closely as possible. It also adds and adapts many tests, to ensure full coverage of the changes.

I now believe supporting ABC mapping views is worth it: it adds little code and broadens the kinds of mapping for which we can pretty print views.

Some choices, like calling repr() only to raise an AttributeError instead of raising it ourselves, I'm not so sure about.

Code and output for pretty printing many kinds of views

import pprint
from collections import OrderedDict, defaultdict, Counter, ChainMap
from collections.abc import Mapping, MappingView, ItemsView, KeysView, ValuesView

ABC_MAPPING_VIEWS = MappingView, ItemsView, KeysView, ValuesView
DICT_CLASSES = OrderedDict, Counter, ChainMap

def main():
    # Simple dictionary with string keys and values
    simple_dict = {"apple": "fruit", "carrot": "vegetable", "pear": "fruit"}

    # Dictionary with various types of keys and values
    mixed_dict = {42: "answer", (2, 3): [1, 2, 3], 'key': {'nested': 'dict'}}

    # Long dictionary with integer keys and string values
    long_dict = {i: f"number_{i}" for i in range(20)}

    # Long recursive dictionary
    recursive_dict = long_dict.copy()
    recursive_dict[20] = recursive_dict

    # Dictionary containing views
    views_dict = {i: cls({i: i}) for i, cls in enumerate(ABC_MAPPING_VIEWS)}
    start = len(ABC_MAPPING_VIEWS)
    views_dict.update({i + start: cls({i: i}).items() for i, cls in enumerate(DICT_CLASSES)})
    int_default_dict = defaultdict(int)
    int_default_dict[0] = 0
    views_dict.update({7: int_default_dict.items()})

    # Ordered dictionary to maintain insertion order
    ordered_dict = OrderedDict([(f"key_{i}", i) for i in range(20)])

    # Default dictionary with a default factory of list
    default_dict = defaultdict(list)
    default_dict['fruits'].append('apple')
    default_dict['fruits'].append('banana')
    default_dict['vegetables'].append('carrot')

    # Counter dictionary to count occurrences
    counter_dict = Counter('abracadabra')

    # ChainMap to search multiple dictionaries
    chain_map = ChainMap(simple_dict, mixed_dict, long_dict)

    # Nested dictionary views
    nested_dict = {'level1': simple_dict, 'level2': {'level2_dict': mixed_dict}}

    # Printing different dictionaries with pprint
    print("Simple Dict Keys:")
    pprint.pprint(simple_dict.keys())
    print("\nMixed Dict Items:")
    pprint.pprint(mixed_dict.items())

    print("\nLong Dict Values:")
    pprint.pprint(long_dict.values(), width=50)

    print("\nOrdered Dict Items:")
    pprint.pprint(ordered_dict.items())

    print("\nDefault Dict Items:")
    pprint.pprint(default_dict.items())

    print("\nCounter Dict Items:")
    pprint.pprint(counter_dict.items())

    print("\nChainMap Items:")
    pprint.pprint(chain_map.items())

    print("\nNested Dict Views:")
    pprint.pprint(nested_dict.keys())
    pprint.pprint(nested_dict.items())
    pprint.pprint(nested_dict.items(), depth=2)

    print("\nRecursive Dict Views:")
    pprint.pprint(recursive_dict.items())

    print("\nABC Mapping Views (should pretty print a repr of the mapping):")
    pprint.pprint(MappingView(recursive_dict))
    pprint.pprint(ItemsView(nested_dict))
    pprint.pprint(KeysView(chain_map))
    pprint.pprint(ValuesView(ordered_dict))

    print("\nDict With Views:")
    pprint.pprint(views_dict.items())

    classes()


def classes():

    class Mappingview_Custom_Repr(MappingView):
        def __repr__(self):
            return '*' * len(
67E6
MappingView.__repr__(self))

    print("\nCustom __repr__ in Subclass:")
    pprint.pprint(Mappingview_Custom_Repr({1: 1}))

    class MyMapping(Mapping):
        def __init__(self, keys=None):
            self._keys = {} if keys is None else dict.fromkeys(keys)

        def __getitem__(self, item):
            return self._keys[item]

        def __len__(self):
            return len(self._keys)

        def __iter__(self):
            return iter(self._keys)

        def __repr__(self):
            return f"{self.__class__.__name__}([{', '.join(map(repr, self._keys.keys()))}])"

    print("\nCustom __repr__ in _mapping:")
    my_mapping = MyMapping(["test", 1])
    pprint.pprint(MappingView(my_mapping))
    pprint.pprint(my_mapping.items())

if __name__ == "__main__":
    main()

Output:

Simple Dict Keys:
dict_keys(['apple', 'carrot', 'pear'])

Mixed Dict Items:
dict_items([(42, 'answer'), ('key', {'nested': 'dict'}), ((2, 3), [1, 2, 3])])

Long Dict Values:
dict_values(['number_0',
 'number_1',
 'number_10',
 'number_11',
 'number_12',
 'number_13',
 'number_14',
 'number_15',
 'number_16',
 'number_17',
 'number_18',
 'number_19',
 'number_2',
 'number_3',
 'number_4',
 'number_5',
 'number_6',
 'number_7',
 'number_8',
 'number_9'])

Ordered Dict Items:
odict_items([('key_0', 0),
 ('key_1', 1),
 ('key_10', 10),
 ('key_11', 11),
 ('key_12', 12),
 ('key_13', 13),
 ('key_14', 14),
 ('key_15', 15),
 ('key_16', 16),
 ('key_17', 17),
 ('key_18', 18),
 ('key_19', 19),
 ('key_2', 2),
 ('key_3', 3),
 ('key_4', 4),
 ('key_5', 5),
 ('key_6', 6),
 ('key_7', 7),
 ('key_8', 8),
 ('key_9', 9)])

Default Dict Items:
dict_items([('fruits', ['apple', 'banana']), ('vegetables', ['carrot'])])

Counter Dict Items:
dict_items([('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)])

ChainMap Items:
ItemsView(ChainMap({'apple': 'fruit', 'carrot': 'vegetable', 'pear': 'fruit'},
         {42: 'answer', 'key': {'nested': 'dict'}, (2, 3): [1, 2, 3]},
         {0: 'number_0',
          1: 'number_1',
          2: 'number_2',
          3: 'number_3',
          4: 'number_4',
          5: 'number_5',
          6: 'number_6',
          7: 'number_7',
          8: 'number_8',
          9: 'number_9',
          10: 'number_10',
          11: 'number_11',
          12: 'number_12',
          13: 'number_13',
          14: 'number_14',
          15: 'number_15',
          16: 'number_16',
          17: 'number_17',
          18: 'number_18',
          19: 'number_19'}))

Nested Dict Views:
dict_keys(['level1', 'level2'])
dict_items([('level1', {'apple': 'fruit', 'carrot': 'vegetable', 'pear': 'fruit'}),
 ('level2',
  {'level2_dict': {42: 'answer',
                   'key': {'nested': 'dict'},
                   (2, 3): [1, 2, 3]}})])
dict_items([('level1', {...}), ('level2', {...})])

Recursive Dict Views:
dict_items([(0, 'number_0'),
 (1, 'number_1'),
 (2, 'number_2'),
 (3, 'number_3'),
 (4, 'number_4'),
 (5, 'number_5'),
 (6, 'number_6'),
 (7, 'number_7'),
 (8, 'number_8'),
 (9, 'number_9'),
 (10, 'number_10'),
 (11, 'number_11'),
 (12, 'number_12'),
 (13, 'number_13'),
 (14, 'number_14'),
 (15, 'number_15'),
 (16, 'number_16'),
 (17, 'number_17'),
 (18, 'number_18'),
 (19, 'number_19'),
 (20,
  {0: 'number_0',
   1: 'number_1',
   2: 'number_2',
   3: 'number_3',
   4: 'number_4',
   5: 'number_5',
   6: 'number_6',
   7: 'number_7',
   8: 'number_8',
   9: 'number_9',
   10: 'number_10',
   11: 'number_11',
   12: 'number_12',
   13: 'number_13',
   14: 'number_14',
   15: 'number_15',
   16: 'number_16',
   17: 'number_17',
   18: 'number_18',
   19: 'number_19',
   20: <Recursion on dict with id=2200369163696>})])

ABC Mapping Views (should pretty print a repr of the mapping):
MappingView({0: 'number_0',
 1: 'number_1',
 2: 'number_2',
 3: 'number_3',
 4: 'number_4',
 5: 'number_5',
 6: 'number_6',
 7: 'number_7',
 8: 'number_8',
 9: 'number_9',
 10: 'number_10',
 11: 'number_11',
 12: 'number_12',
 13: 'number_13',
 14: 'number_14',
 15: 'number_15',
 16: 'number_16',
 17: 'number_17',
 18: 'number_18',
 19: 'number_19',
 20: <Recursion on dict with id=2200369163696>})
ItemsView({'level1': {'apple': 'fruit', 'carrot': 'vegetable', 'pear': 'fruit'},
 'level2': {'level2_dict': {42: 'answer',
                            'key': {'nested': 'dict'},
                            (2, 3): [1, 2, 3]}}})
KeysView(ChainMap({'apple': 'fruit', 'carrot': 'vegetable', 'pear': 'fruit'},
         {42: 'answer', 'key': {'nested': 'dict'}, (2, 3): [1, 2, 3]},
         {0: 'number_0',
          1: 'number_1',
          2: 'number_2',
          3: 'number_3',
          4: 'number_4',
          5: 'number_5',
          6: 'number_6',
          7: 'number_7',
          8: 'number_8',
          9: 'number_9',
          10: 'number_10',
          11: 'number_11',
          12: 'number_12',
          13: 'number_13',
          14: 'number_14',
          15: 'number_15',
          16: 'number_16',
          17: 'number_17',
          18: 'number_18',
          19: 'number_19'}))
ValuesView(OrderedDict([('key_0', 0),
             ('key_1', 1),
             ('key_2', 2),
             ('key_3', 3),
             ('key_4', 4),
             ('key_5', 5),
             ('key_6', 6),
             ('key_7', 7),
             ('key_8', 8),
             ('key_9', 9),
             ('key_10', 10),
             ('key_11', 11),
             ('key_12', 12),
             ('key_13', 13),
             ('key_14', 14),
             ('key_15', 15),
             ('key_16', 16),
             ('key_17', 17),
             ('key_18', 18),
             ('key_19', 19)]))

Dict With Views:
dict_items([(0, MappingView({0: 0})),
 (1, ItemsView({1: 1})),
 (2, KeysView({2: 2})),
 (3, ValuesView({3: 3})),
 (4, odict_items([(0, 0)])),
 (5, dict_items([(1, 1)])),
 (6, ItemsView(ChainMap({2: 2}))),
 (7, dict_items([(0, 0)]))])

Custom __repr__ in Subclass:
*******************************

Custom __repr__ in _mapping:
MappingView(MyMapping(['test', 1]))
ItemsView(MyMapping(['test', 1]))

@devdanzin devdanzin marked this pull request as ready for review April 26, 2024 11:59
Copy link
Contributor
@davidlowryduda davidlowryduda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm that the changes perform as indicated. This all works well.

I think the only oddity is the recursive printing, but I don't think that should be a block. Instead, if someone has a great idea for how to better handle recursively printing blocks (or, for example, asking if this is useful enough to warrant more attention), then they can do that later.

@python-cla-bot
Copy link
python-cla-bot bot commented Apr 18, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

Copy link
Contributor
@AlexKautz AlexKautz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Hello from PyCon's CPython sprint!]
I looked it over and did some manual testing and is working correctly. Good work!

@gpshead gpshead self-assigned this May 20, 2025
@gpshead gpshead added 3.15 new features, bugs and security fixes and removed 3.13 bugs and security fixes labels May 20, 2025
Copy link
Member
@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed (pairing) - this is in a good state and is an improvement over existing pprint behavior.

@gpshead gpshead merged commit c7f8e70 into python:main May 20, 2025
44 of 47 checks passed
@merwok
Copy link
Member
merwok commented May 20, 2025

A small thing: github unhelpfully defaults merge commit message to the concatenation of all commits in the branch. One needs to delete it before merging. The browser extension Refined Github helps with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.15 new features, bugs and security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0