8000 configparser API cleanup: default values now sensible, slightly incom… · python/cpython@b25a791 · GitHub
[go: up one dir, main page]
Skip to content

Commit b25a791

Browse files
ambvambv
committed
configparser API cleanup: default values now sensible, slightly incompatible.
Backwards compatible alternative values possible as documented. Done by Łukasz Langa, approved by Raymond and Fred.
1 parent ed16bf4 commit b25a791

File tree

3 files changed

+117
-64
lines changed

3 files changed

+117
-64
lines changed

‎Doc/library/configparser.rst‎

Lines changed: 89 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -230,21 +230,18 @@ may be treated as parts of multiline values or ignored.
230230

231231
Configuration files may include comments, prefixed by specific
232232
characters (``#`` and ``;`` by default [1]_). Comments may appear on
233-
their own on an otherwise empty line, or may be entered on lines holding
234-
values or section names. In the latter case, they need to be preceded
235-
by a whitespace character to be recognized as a comment. For backwards
236-
compatibility, by default only ``;`` starts an inline comment, while
237-
``#`` does not [1]_.
233+
their own on an otherwise empty line, possibly indented. [1]_
238234

239235
For example:
240236

241237
.. code-block:: ini
242238
243239
[Simple Values]
244-
key: value
245-
spaces in keys: allowed
246-
spaces in values: allowed as well
247-
you can also use = to delimit keys from values
240+
key=value
241+
spaces in keys=allowed
242+
spaces in values=allowed as well
243+
spaces around the delimiter = obviously
244+
you can also use : to delimit keys from values
248245
249246
[All Values Are Strings]
250247
values like this: 1000000
@@ -261,12 +258,14 @@ For example:
261258
key_without_value
262259
empty string value here =
263260
264-
[You can use comments] ; after a useful line
265-
; in an empty line
266-
after: a_value ; here' 8000 s another comment
267-
inside: a ;comment
268-
multiline ;comment
269-
value! ;comment
261+
[You can use comments]
262+
# like this
263+
; or this
264+
265+
# By default only in an empty line.
266+
# Inline comments can be harmful because they prevent users
267+
# from using the delimiting characters as parts of values.
268+
# That being said, this can be customized.
270269
271270
[Sections Can Be Indented]
272271
can_values_be_as_well = True
@@ -509,7 +508,8 @@ the :meth:`__init__` options:
509508
... skip-external-locking
510509
... old_passwords = 1
511510
... skip-bdb
512-
... skip-innodb # we don't need ACID today
511+
... # we don't need ACID today
512+
... skip-innodb
513513
... """
514514
>>> config = configparser.ConfigParser(allow_no_value=True)
515515
>>> config.read_string(sample_config)
@@ -536,28 +536,78 @@ the :meth:`__init__` options:
536536
See also the *space_around_delimiters* argument to
537537
:meth:`ConfigParser.write`.
538538

539-
* *comment_prefixes*, default value: ``_COMPATIBLE`` (``'#'`` valid on empty
540-
lines, ``';'`` valid also on non-empty lines)
539+
* *comment_prefixes*, default value: ``('#', ';')``
541540

542-
Comment prefixes are strings that indicate the start of a valid comment
543-
within a config file. The peculiar default value allows for comments starting
544-
with ``'#'`` or ``';'`` but only the latter can be used in a non-empty line.
545-
This is obviously dictated by backwards compatibiliy. A more predictable
546-
approach would be to specify prefixes as ``('#', ';')`` which will allow for
547-
both prefixes to be used in non-empty lines.
541+
* *inline_comment_prefixes*, default value: ``None``
548542

549-
Please note that config parsers don't support escaping of comment prefixes so
550-
leaving characters out of *comment_prefixes* is a way of ensuring they can be
551-
used as parts of keys or values.
543+
Comment prefixes are strings that indicate the start of a valid comment within
544+
a config file. *comment_prefixes* are used only on otherwise empty lines
545+
(optionally indented) whereas *inline_comment_prefixes* can be used after
546+
every valid value (e.g. section names, options and empty lines as well). By
547+
default inline comments are disabled and ``'#'`` and ``';'`` are used as
548+
prefixes for whole line comments.
552549

553-
* *strict*, default value: ``False``
550+
.. versionchanged:: 3.2
551+
In previous versions of :mod:`configparser` behaviour matched
552+
``comment_prefixes=('#',';')`` and ``inline_comment_prefixes=(';',)``.
554553

555-
If set to ``True``, the parser will not allow for any section or option
554+
Please note that config parsers don't support escaping of comment prefixes so
555+
using *inline_comment_prefixes* may prevent users from specifying option
556+
values with characters used as comment prefixes. When in doubt, avoid setting
557+
*inline_comment_prefixes*. In any circumstances, the only way of storing
558+
comment prefix characters at the beginning of a line in multiline values is to
559+
interpolate the prefix, for example::
560+
561+
>>> from configparser import ConfigParser, ExtendedInterpolation
562+
>>> parser = ConfigParser(interpolation=ExtendedInterpolation())
563+
>>> # the default BasicInterpolation could be used as well
564+
>>> parser.read_string("""
565+
... [DEFAULT]
566+
... hash = #
567+
...
568+
... [hashes]
569+
... shebang =
570+
... ${hash}!/usr/bin/env python
571+
... ${hash} -*- coding: utf-8 -*-
572+
...
573+
... extensions =
574+
... enabled_extension
575+
... another_extension
576+
... #disabled_by_comment
577+
... yet_another_extension
578+
...
579+
... interpolation not necessary = if # is not at line start
580+
... even in multiline values = line #1
581+
... line #2
582+
... line #3
583+
... """)
584+
>>> print(parser['hashes']['shebang'])
585+
586+
#!/usr/bin/env python
587+
# -*- coding: utf-8 -*-
588+
>>> print(parser['hashes']['extensions'])
589+
590+
enabled_extension
591+
another_extension
592+
yet_another_extension
593+
>>> print(parser['hashes']['interpolation not necessary'])
594+
if # is not at line start
595+
>>> print(parser['hashes']['even in multiline values'])
596+
line #1
597+
line #2
598+
line #3
599+
600+
* *strict*, default value: ``True``
601+
602+
When set to ``True``, the parser will not allow for any section or option
556603
duplicates while reading from a single source (using :meth:`read_file`,
557-
:meth:`read_string` or :meth:`read_dict`). The default is ``False`` only
558-
because of backwards compatibility reasons. It is recommended to use strict
604+
:meth:`read_string` or :meth:`read_dict`). It is recommended to use strict
559605
parsers in new applications.
560606

607+
.. versionchanged:: 3.2
608+
In previous versions of :mod:`configparser` behaviour matched
609+
``strict=False``.
610+
561611
* *empty_lines_in_values*, default value: ``True``
562612

563613
In config parsers, values can span multiple lines as long as they are
@@ -575,7 +625,6 @@ the :meth:`__init__` options:
575625
576626
this = is still a part of the multiline value of 'key'
577627
578-
579628
This can be especially problematic for the user to see if she's using a
580629
proportional font to edit the file. That is why when your application does
581630
not need values with empty lines, you should consider disallowing them. This
@@ -603,8 +652,7 @@ the :meth:`__init__` options:
603652
interpolation completely, ``ExtendedInterpolation()`` provides a more
604653
advanced variant inspired by ``zc.buildout``. More on the subject in the
605654
`dedicated documentation section <#interpolation-of-values>`_.
606-
607-
.. note:: :class:`RawConfigParser` is using ``None`` by default.
655+
:class:`RawConfigParser` has a default value of ``None``.
608656

609657

610658
More advanced customization may be achieved by overriding default values of
@@ -769,7 +817,7 @@ interpolation if an option used is not defined elsewhere. ::
769817
ConfigParser Objects
770818
--------------------
771819

772-
.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict, allow_no_value=False, delimiters=('=', ':'), comment_prefixes=_COMPATIBLE, strict=False, empty_lines_in_values=True, default_section=configparser.DEFAULTSECT, interpolation=BasicInterpolation())
820+
.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict, allow_no_value=False, delimiters=('=', ':'), comment_prefixes=('#', ';'), inline_comment_prefixes=None, strict=True, empty_lines_in_values=True, default_section=configparser.DEFAULTSECT, interpolation=BasicInterpolation())
773821

774822
The main configuration parser. When *defaults* is given, it is initialized
775823
into the dictionary of intrinsic defaults. When *dict_type* is given, it
@@ -778,12 +826,15 @@ ConfigParser Objects
778826

779827
When *delimiters* is given, it is used as the set of substrings that
780828
divide keys from values. When *comment_prefixes* is given, it will be used
781-
as the set of substrings that prefix comments in a line, both for the whole
829+
as the set of substrings that prefix comments in otherwise empty lines.
830+
Comments can be indented. When *inline_comment_prefixes* is given, it will be
831+
used as the set of substrings that prefix comments in non-empty lines.
832+
782833
line and inline comments. For backwards compatibility, the default value for
783834
*comment_prefixes* is a special value that indicates that ``;`` and ``#`` can
784835
start whole line comments while only ``;`` can start inline comments.
785836

786-
When *strict* is ``True`` (default: ``False``), the parser won't allow for
837+
When *strict* is ``True`` (the default), the parser won't allow for
787838
any section or option duplicates while reading from a single source (file,
788839
string or dictionary), raising :exc:`DuplicateSectionError` or
789840
:exc:`DuplicateOptionError`. When *empty_lines_in_values* is ``False``
@@ -1043,7 +1094,7 @@ ConfigParser Objects
10431094
RawConfigParser Objects
10441095
-----------------------
10451096

1046-
.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict, allow_no_value=False, delimiters=('=', ':'), comment_prefixes=_COMPATIBLE, strict=False, empty_lines_in_values=True, default_section=configaparser.DEFAULTSECT, interpolation=None)
1097+
.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict, allow_no_value=False, delimiters=('=', ':'), comment_prefixes=('#', ';'), inline_comment_prefixes=None, strict=True, empty_lines_in_values=True, default_section=configaparser.DEFAULTSECT, interpolation=None)
10471098

10481099
Legacy variant of the :class:`ConfigParser` with interpolation disabled
10491100
by default and unsafe ``add_section`` and ``set`` methods.

‎Lib/configparser.py‎

Lines changed: 19 additions & 23 deletions
< BD9E /tr>
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,9 @@
1515
methods:
1616
1717
__init__(defaults=None, dict_type=_default_dict, allow_no_value=False,
18-
delimiters=('=', ':'), comment_prefixes=_COMPATIBLE,
19-
strict=False, empty_lines_in_values=True):
18+
delimiters=('=', ':'), comment_prefixes=('#', ';'),
19+
inline_comment_prefixes=None, strict=True,
20+
empty_lines_in_values=True):
2021
Create the parser. When `defaults' is given, it is initialized into the
2122
dictionary or intrinsic defaults. The keys must be strings, the values
2223
must be appropriate for %()s string interpolation.
@@ -29,11 +30,15 @@
2930
that divide keys from values.
3031
3132
When `comment_prefixes' is given, it will be used as the set of
32-
substrings that prefix comments in a line.
33+
substrings that prefix comments in empty lines. Comments can be
34+
indented.
35+
36+
When `inline_comment_prefixes' is given, it will be used as the set of
37+
substrings that prefix comments in non-empty lines.
3338
3439
When `strict` is True, the parser won't allow for any section or option
3540
duplicates while reading from a single source (file, string or
36-
dictionary). Default is False.
41+
dictionary). Default is True.
3742
3843
When `empty_lines_in_values' is False (default: True), each empty line
3944
marks the end of an option. Otherwise, internal empty lines of
@@ -340,11 +345,6 @@ def __init__(self, filename, lineno, line):
340345
self.args = (filename, lineno, line)
341346

342347

343-
# Used in parsers to denote selecting a backwards-compatible inline comment
344-
# character behavior (; and # are comments at the start of a line, but ; only
345-
# inline)
346-
_COMPATIBLE = object()
347-
348348
# Used in parser getters to indicate the default behaviour when a specific
349349
# option is not found it to raise an exception. Created to enable `None' as
350350
# a valid fallback value.
@@ -592,8 +592,8 @@ class RawConfigParser(MutableMapping):
592592

593593
def __init__(self, defaults=None, dict_type=_default_dict,
594594
allow_no_value=False, *, delimiters=('=', ':'),
595-
comment_prefixes=_COMPATIBLE, strict=False,
596-
empty_lines_in_values=True,
595+
comment_prefixes=('#', ';'), inline_comment_prefixes=None,
596+
strict=True, empty_lines_in_values=True,
597597
default_section=DEFAULTSECT,
598598
interpolation=_UNSET):
599599

@@ -616,12 +616,8 @@ def __init__(self, defaults=None, dict_type=_default_dict,
616616
else:
617617
self._optcre = re.compile(self._OPT_TMPL.format(delim=d),
618618
re.VERBOSE)
619-
if comment_prefixes is _COMPATIBLE:
620-
self._startonly_comment_prefixes = ('#',)
621-
self._comment_prefixes = (';',)
622-
else:
623-
self._startonly_comment_prefixes = ()
624-
self._comment_prefixes = tuple(comment_prefixes or ())
619+
self._comment_prefixes = tuple(comment_prefixes or ())
620+
self._inline_comment_prefixes = tuple(inline_comment_prefixes or ())
625621
self._strict = strict
626622
self._allow_no_value = allow_no_value
627623
self._empty_lines_in_values = empty_lines_in_values
@@ -989,18 +985,18 @@ def _read(self, fp, fpname):
989985
indent_level = 0
990986
e = None # None, or an exception
991987
for lineno, line in enumerate(fp, start=1):
992-
# strip full line comments
993988
comment_start = None
994-
for prefix in self._startonly_comment_prefixes:
995-
if line.strip().startswith(prefix):
996-
comment_start = 0
997-
break
998989
# strip inline comments
999-
for prefix in self._comment_prefixes:
990+
for prefix in self._inline_comment_prefixes:
1000991
index = line.find(prefix)
1001992
if index == 0 or (index > 0 and line[index-1].isspace()):
1002993
comment_start = index
1003994
break
995+
# strip full line comments
996+
for prefix in self._comment_prefixes:
997+
if line.strip().startswith(prefix):
998+
comment_start = 0
999+
break
10041000
value = line[:comment_start].strip()
10051001
if not value:
10061002
if self._empty_lines_in_values:

‎Lib/test/test_cfgparser.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ class CfgParserTestCaseClass(unittest.TestCase):
2929
allow_no_value = False
3030
delimiters = ('=', ':')
3131
comment_prefixes = (';', '#')
32+
inline_comment_prefixes = (';', '#')
3233
empty_lines_in_values = True
3334
dict_type = configparser._default_dict
3435
strict = False
@@ -41,6 +42,7 @@ def newconfig(self, defaults=None):
4142
allow_no_value=self.allow_no_value,
4243
delimiters=self.delimiters,
4344
comment_prefixes=self.comment_prefixes,
45+
inline_comment_prefixes=self.inline_comment_prefixes,
4446
empty_lines_in_values=self.empty_lines_in_values,
4547
dict_type=self.dict_type,
4648
strict=self.strict,
@@ -812,6 +814,7 @@ def test_set_malformatted_interpolation(self):
812814
class ConfigParserTestCaseNonStandardDelimiters(ConfigParserTestCase):
813815
delimiters = (':=', '$')
814816
comment_prefixes = ('//', '"')
817+
inline_comment_prefixes = ('//', '"')
815818

816819
class ConfigParserTestCaseNonStandardDefaultSection(ConfigParserTestCase):
817820
default_section = 'general'
@@ -888,10 +891,12 @@ def test_set_nonstring_types(self):
888891
class RawConfigParserTestCaseNonStandardDelimiters(RawConfigParserTestCase):
889892
delimiters = (':=', '$')
890893
comment_prefixes = ('//', '"')
894+
inline_comment_prefixes = ('//', '"')
891895

892-
class RawConfigParserTestSambaConf(BasicTestCase):
896+
class RawConfigParserTestSambaConf(CfgParserTestCaseClass):
893897
config_class = configparser.RawConfigParser
894-
comment_prefixes = ('#', ';', '//', '----')
898+
comment_prefixes = ('#', ';', '----')
899+
inline_comment_prefixes = ('//',)
895900
empty_lines_in_values = False
896901

897902
def test_reading(self):
@@ -1074,7 +1079,8 @@ def test_sorted(self):
10741079

10751080
class CompatibleTestCase(CfgParserTestCaseClass):
10761081
config_class = configparser.RawConfigParser
1077-
comment_prefixes = configparser._COMPATIBLE
1082+
comment_prefixes = '#;'
1083+
inline_comment_prefixes = ';'
10781084

10791085
def test_comment_handling(self):
10801086
config_string = textwrap.dedent("""\

0 commit comments

Comments
 (0)
0