8000 bpo-30181: parse docstring using pydoc by brianmay · Pull Request #1505 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-30181: parse docstring using pydoc #1505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 23 additions & 11 deletions Lib/unittest/case.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import difflib
import logging
import pprint
import pydoc
import re
import warnings
import collections
Expand Down Expand Up @@ -338,6 +339,25 @@ def __exit__(self, exc_type, exc_value, tb):
.format(logging.getLevelName(self.level), self.logger.name))


def short_description_from_docstring(text):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function name should start with an underscore since it's not meant to be part of the public API. Also, I think a shorter name should be used like _make_short_description(docstring) or _get_short_description(docstring). Information about the arguments can be gotten from the argument names or function docstring.

"""
Return the test case short description from a docstring.

Ths docstring is parsed by the standard `pydoc.splitdoc` function
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an implementation detail. Both short_description_from_docstring and pydoc.splitdoc are not public API, therefore the docstring is interested only for readers of these sources. It can be just a comment. But there is no much sense to just repeat the code by words in a docstring or comment.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There appears to be different ideas at play here surrounding the use docstrings. With one view being docstrings should only document the public API, so users of the API know how to use the package. The other view being docstrings are equally applicable for internal functions, which should have a well defined API for internal use, so internal programmers can tell what the function does without peering into the source. Any external references to help justify one or the other side of this argument appreciated (I have no opinions on this yet myself).

Copy link
Author
@brianmay brianmay May 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have also had the out of band comment from Ben saying just because it isn't a public API doesn't make the docstring any less useful. Since a docstring will be available to anyone introspecting the object whether private or public

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstrings of internal functions are not forbidden, they are just not necessary. Comments would be enough (see PEP 8).

In any case neither comment, nor docstring shouldn't just repeat the code. They should describe what function does, but not how it does this. Otherwise any change of the code will make a docstring or a comment incorrect.

Note also that the function name itself serves the documenting function.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

def short_description_from_docstring(text):
    # Return the summary line from a docstring.
    # If there is no summary line, return None.
    ...

The use of the word "summary line" comes straight from PEP-257.

into (`synopsis`, `long_description`).

If there is no `synopsis`, return None.
"""
if text:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be cleaner to use a guard clause and just write (and with no comment):

if not text:
    return None

(synopsis, long_description) = pydoc.splitdoc(text)
synopsis = synopsis.strip()
else:
# The text is either an empty string, or some other false value.
synopsis = None

return synopsis


class TestCase(object):
"""A class whose instances are single test cases.

Expand Down Expand Up @@ -463,15 +483,8 @@ def defaultTestResult(self):
return result.TestResult()

def shortDescription(self):
"""Returns a one-line description of the test, or None if no
description has been provided.

The default implementation of this method returns the first line of
the specified test method's docstring.
"""
doc = self._testMethodDoc
return doc and doc.split("\n")[0].strip() or None

""" Return a one-line description of the test, if any; otherwise None. """
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shortDescription() is a public API. Your change removes useful information from its docstring.

Copy link
Author
@brianmay brianmay May 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that the previous description is entirely relevant. Also does saying "The default implementation" add any useful information? How about:

def shortDescription(self):
    """Returns a one-line description of the test, or None if no
    description has been provided.

    This method returns the summary line of
    the specified test method's docstring, as per PEP-257.
    """
    ...

return short_description_from_docstring(self._testMethodDoc)

def id(self):
return "%s.%s" % (strclass(self.__class__), self._testMethodName)
Expand Down Expand Up @@ -1395,8 +1408,7 @@ def __repr__(self):
def shortDescription(self):
if self._description is not None:
return self._description
doc = self._testFunc.__doc__
return doc and doc.split("\n")[0].strip() or None
return short_description_from_docstring(self._testFunc.__doc__)


class _SubTest(TestCase):
Expand Down
42 changes: 42 additions & 0 deletions Lib/unittest/test/test_case.py
Original file line number Diff line number Diff line change
Expand Up @@ -596,6 +596,48 @@ def testShortDescriptionWithMultiLineDocstring(self):
'Tests shortDescription() for a method with a longer '
'docstring.')

@unittest.skipIf(
sys.flags.optimize >= 2,
"Docstrings are omitted with -O2 and above")
def testShortDescriptionWithSurroundingSpaceOneLineDocstring(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is enough to just add spaces or newlines in the two preceding tests.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, not sure I understand. You appear to be saying that we should use the same docstring as for the testShortDescriptionWithOneLineDocstring test but with extra white space. However it is a different test - doesn't that mean it needs a different doc string to describe how it is different from the previous docstring?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that adding spaces at the start and end of the docstring of testShortDescriptionWithOneLineDocstring and adding a newline and indentation at the start of the docstring of testShortD 9E7A escriptionWithMultiLineDocstring will cover cases tested by new testes. No new tests will be needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, OK, I think I understand now. So to clarify, you feel there is no need to have a separate test for the no-leading/trailing whitespace case?

""" Surrounding space should be stripped to get the shortDescription. """
expected_description = (
"Surrounding space should be stripped"
" to get the shortDescription.")
self.assertEqual(self.shortDescription(), expected_description)

@unittest.skipIf(
sys.flags.optimize >= 2,
"Docstrings are omitted with -O2 and above")
def testShortDescriptionWithSurroundingNewlineOneLineDocstring(self):
"""
Surrounding newlines should be stripped to get the shortDescription.
"""
expected_description = (
"Surrounding newlines should be stripped"
" to get the shortDescription.")
self.assertEqual(self.shortDescription(), expected_description)

@unittest.skipIf(
sys.flags.optimize >= 2,
"Docstrings are omitted with -O2 and above")
def testShortDescriptionWithSurroundingNewlineMultiLineDocstring(self):
"""
Surrounding newlines should be stripped to get the shortDescription.

The specification of how docstring space should be parsed is at
https://www.python.org/dev/peps/pep-0257/#handling-docstring-indentation
which requires that “Blank lines should be removed from the
beginning and end of the docstring.”

The PEP 257 algorithm is implemented by `pydoc.splitdoc`.

"""
expected_description = (
"Surrounding newlines should be stripped"
" to get the shortDescription.")
self.assertEqual(self.shortDescription(), expected_description)

def testAddTypeEqualityFunc(self):
class SadSnake(object):
"""Dummy class for test_addTypeEqualityFunc."""
Expand Down
58 changes: 50 additions & 8 deletions Lib/unittest/test/test_functiontestcase.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,23 +126,65 @@ def test_id(self):

self.assertIsInstance(test.id(), str)

# "Returns a one-line description of the test, or None if no description
# has been provided. The default implementation of this method returns
# the first line of the test method's docstring, if available, or None."
def test_shortDescription__no_docstring(self):
def test_shortDescription__no_description_no_docstring(self):
""" Should return None by default for shortDescription. """
test = unittest.FunctionTestCase(lambda: None)

self.assertEqual(test.shortDescription(), None)

# "Returns a one-line description of the test, or None if no description
# has been provided. The default implementation of this method returns
# the first line of the test method's docstring, if available, or None."
def test_shortDescription__singleline_docstring(self):
def test_shortDescription__singleline_description(self):
""" Should use the specified description for shortDescription. """
desc = "this tests foo"
test = unittest.FunctionTestCase(lambda: None, description=desc)

self.assertEqual(test.shortDescription(), "this tests foo")

def test_shortDescription__no_description_singleline_docstring(self):
""" Should use the function docstring for the shortDescription. """
test_function = (lambda: None)
test_function.__doc__ = """Should use the function docstring."""
test = unittest.FunctionTestCase(test_function)
expected_description = "Should use the function docstring."
self.assertEqual(test.shortDescription(), expected_description)

def test_shortDescription__singleline_docstring_space_surrounded(self):
""" Surrounding space should be stripped to get the shortDescription. """
test_function = (lambda: None)
test_function.__doc__ = """ Surrounding space should be stripped. """
test = unittest.FunctionTestCase(test_function)
expected_description = "Surrounding space should be stripped."
self.assertEqual(test.shortDescription(), expected_description)

def test_shortDescription__singleline_docstring_newline_surrounded(self):
"""
Surrounding newlines should be stripped to get the shortDescription.
"""
test_function = (lambda: None)
test_function.__doc__ = """
Surrounding newlines should be stripped.
"""
test = unittest.FunctionTestCase(test_function)
expected_description = "Surrounding newlines should be stripped."
self.assertEqual(test.shortDescription(), expected_description)

def test_shortDescription__multiline_docstring_newline_surrounded(self):
"""
Surrounding newlines should be stripped to get the shortDescription.
"""
test_function = (lambda: None)
test_function.__doc__ = """
Surrounding newlines should be stripped.

The specification of how docstring space should be parsed is at
https://www.python.org/dev/peps/pep-0257/#handling-docstring-indentation
which requires that “Blank lines should be removed from the
beginning and end of the docstring.”

"""
test = unittest.FunctionTestCase(test_function)
expected_description = "Surrounding newlines should be stripped."
self.assertEqual(test.shortDescription(), expected_description)


if __name__ == "__main__":
unittest.main()
2911
0