Open
Description
[edit by @encukou, May 2024] The coverage report and checklist are outdated. Run coverage locally before contributing.
This bug is going to be used to track work in a other bugs to improve the C-level coverage of the CPython test suite.
There is a set of baseline coverage results on main [edit: outdated, see above] that can be used to find coverage gaps.
The plan, discussed on discuss.python.org is as follows:
- Read through the coverage report and record any notable gaps in the checklist below. The goal is not 100% coverage, and each area of improvement will probably require some judgement calls. For example, covering all cases where memory exhaustion can occur is probably not worth the effort. On the other hand, detailed coverage in the eval loop may be worth the effort.
- When someone has "read through" a particular source file and added created subitems for any interesting gaps, they should check it off on the list below and add links to any issues created.
Related work:
There is related work to publish coverage results from CPython on a regular basis, but this issue is concerned with using those results to actually reduce our gaps in coverage.
List of source files:
-
Include/internal/pycore_asdl.h -
Include/internal/pycore_bitutils.h -
Include/internal/pycore_call.h -
Include/internal/pycore_code.h -
Include/internal/pycore_frame.h -
Include/internal/pycore_moduleobject.h -
Include/internal/pycore_object.h -
Include/internal/pycore_pymath.h -
Include/internal/pycore_pymem.h -
Include/internal/pycore_pystate.h -
Include/object.h -
Include/pydtrace.h -
Objects/abstract.c- Buffer related functions:
PyBuffer_FromContiguous,PyObject_CopyData,PyBuffer_FillContiguousStrides -
PyNumber_Checkdoesn't testcomplex -
PySequence_RepeatandPySequence_InPlaceRepeathave no coverage -
PySequence_SetItemwith a negative index is untested -
PySequence_SetSliceandPySequence_DelSliceare untested -
PyMapping_HasKeyandPyMapping_HasKeyStringare untested
- Buffer related functions:
-
Objects/accu.c -
Objects/boolobject.c -
Objects/bytearrayobject.c -
Objects/bytes_methods.c -
Objects/bytesobject.c-
InIt's only the case where the underlying libc is broken that isn't tested.PyBytes_FromFormatV, the special handling of%pisn't tested. - gh-94808: Improve coverage of _PyBytes_FormatEx #95895
-
-
Objects/call.c-
PyEval_CallObjectWithKeywordshas no coverage -
_PyObject_CallMethodId_SizeThas no coverage
-
-
Objects/capsule.c -
Objects/cellobject.c -
Objects/classobject.c -
Objects/codeobject.c -
Objects/complexobject.c -
Objects/descrobject.c -
Objects/dictobject.c- In
dictresizeconvert split table into new combined table" is uncovered. -
_PyDict_GetItemHinthas no coverage
- In
-
Objects/enumobject.c -
Objects/exceptions.c -
Objects/fileobject.c-
PyFile_FromFdhas no coverage -
PyFile_GetLineoverbytesinput has no coverage
-
-
Objects/floatobject.c -
Objects/frameobject.c- frame_setlineno has poor coverage in its helper functions get_arg and mark_stacks.
- _PyFrame_GetState has a switch statement where only the default case is covered.
- In _PyFrame_FastToLocalsWithError there is no test that exercises the COPY_FREE_VARS case.
-
Objects/funcobject.c- A bunch of API is untested:
PyFunction_GetCode,PyFunction_GetGlobals,PyFunction_GetModule,PyFunction_GetDefaults,PyFunction_SetDefaults,PyFunction_GetKwDefaults,PyFunction_SetKwDefaults,PyFunction_GetClosure,PyFunction_SetClosure,PyFunction_GetAnnotations,PyFunction_SetAnnotations\ - gh-94808: cover
PyFunction_GetDefaultsandPyFunction_SetDefaults#98449 - [3.11] gh-94808: Cover
PyFunction_GetCode,PyFunction_GetGlobals,… #98317
- A bunch of API is untested:
-
Objects/genericaliasobject.c -
Objects/genobject.c-
gen_new_with_qualnameand APIPyGen_NewWithQualNameandPyGen_Newhave no coverage. -
PyCoro_Newhas no coverage -
PyAsyncGen_Newhas no coverage -
async_gen_athrow_sendhas poor coverage
-
-
Objects/interpreteridobject.c -
Objects/iterobject.c -
Objects/listobject.c -
Objects/longobject.c-
_PyLong_Sing_t_Converterhas no coverage -
long_format_binarydoesn't test outputting to UCS2 or UCS4 -
int_bit_length_implandint_bit_count_impldoesn't cover the case where expression overflows
-
-
Objects/memoryobject.c-
init_sliceis not well-covered
-
-
Objects/methodobject.c -
Objects/moduleobject.c-
PyModule_GetFilenamehas no coverage
-
-
Objects/namespaceobject.c -
Objects/object.c-
PyObject_Printhas no coverage -
PyObject_Bytesdoes not test the case where there is a__bytes__ - gh-94808: Add test coverage for PyObject_HasAttrString #96627
-
PyObject_SetAttrStringdoesn't test when object has atp_setattr -
PyObject_GetAttrStringdoesn't test when object has atp_getattr -
_PyObject_LookupAttrdoesn't test when object has atp_getattr
-
-
Objects/obmalloc.c -
Objects/odictobject.c -
Objects/picklebufobject.c-
PyPickleBuffer_FromObject,PyPickleBuffer_Releasehas no coverage
-
-
Objects/rangeobject.c -
Objects/setobject.c -
Objects/sliceobject.c-
PySlice_GetIndices/PySlice_GetIndicesExhas no coverage
-
-
Objects/stringlib/codecs.h -
Objects/stringlib/count.h -
Objects/stringlib/ctype.h -
Objects/stringlib/eq.h -
Objects/stringlib/fastsearch.h -
Objects/stringlib/find.h -
Objects/stringlib/find_max_char.h -
Objects/stringlib/join.h -
Objects/stringlib/localeutil.h -
Objects/stringlib/partition.h -
Objects/stringlib/replace.h -
Objects/stringlib/split.h -
Objects/stringlib/transmogrify.h -
Objects/stringlib/undef.h -
Objects/stringlib/unicode_format.h -
Objects/structseq.c -
Objects/tupleobject.c -
Objects/typeobject.c-
wrap_sq_setitemhas no coverage
-
-
Objects/unicodectype.c -
Objects/unicodeobject.c-
xmlcharrefreplacedoesn't test for codepoints < 100 (This seems almost impossible to occur). -
resize_inplacehas no coverage -
unicode_kind_namewhen!PyUnicode_IS_COMPACTisn't covered -- low priority used by consistency check only -
unicode_write_cstrdoesn't test writing into UCS2 or UCS4 - gh-94808: Cover
%pinPyUnicode_FromFormat#96677 -
PyUnicode_AsDecodedObject,PyUnicode_AsDecodedUnicode,PyUnicode_AsEncodedObject,PyUnicode_AsEncodedUnicodehas no coverage -
_Py_DecodeUTF8Exand_Py_EncodeUTF8Exhas no coverage forerror == surrogateescape -
PyUnicode_BuildEncodingMapdoesn't handle theneed_dictcase -
ucs1lib_find_sliceanducs1lib_rfind_slicearen't covered. -
PyUnicode_Counthas no coverage - gh-94808: Cover
str.rsplitfor UCS1, UCS2 or UCS4 #98228 -
PyUnicode_CompareWithASCIIStringhas no coverage for comparing with UCS2 or UCS4 -
_PyUnicode_EqualToASCIIIdhas no coverage
-
-
Objects/unicodetype_db.h -
Objects/unionobject.c -
Objects/weakrefobject.c -
Parser/action_helpers.c-
_PyPegen_set_expr_contextdoesn't cover "starred kind" -
_PyPegen_get_expr_nameswitch statement coverage is non-exhaustive
-
-
Parser/myreadline.c(N/A Windows-only) -
Parser/parser.c -
Parser/peg_api.c -
Parser/pegen.c -
Parser/pegen.h -
Parser/pegen_errors.c -
Parser/string_parser.c -
Parser/tokenizer.c- [coverage] tokenizer.c: valid_utf8 #94823
- tokenizer.c seems to have no coverage for a few functions related to interactive usage, e.g. tok_underflow_interactive and tok_concatenate_interactive_newline.
-
Python/Python-ast.c -
Python/Python-tokenize.c -
Python/_warnings.c-
show_warningdoesn't cover the case where there is asourceline. -
PyErr_WarnExplicithas no coverage
-
-
Python/asdl.c -
Python/ast.c-
ensure_literal_*functions aren't covered -
validate_pattern_match_valuedoesn't cover all elements of switch
-
-
Python/ast_opt.c-
check_complexitydoesn't cover thefrozensetcase -
ast_foldbodyisn't covered
-
-
Python/ast_unparse.c -
Python/bltinmodule.c -
Python/bootstrap_hash.c -
Python/ceval.c-
PyEval_AquireLockandPyEval_ReleaseLockare uncovered - gh-94808: Coverage: __build_class__ inside a non-dict __builtins__ #95932
-
STORE_ATTR_WITH_HINTdoesn't cover the case where the dictionary doesn't have Unicode keys -
CALL_FUNCTION_EXdoesn't cover the case where kwargs is not an exact dict -
PyEval_EvalCodeExdoesn't cover the case where kwargs are passed in -
PyEval_GetFramehas no coverage - gh-94808: Cover
PyEval_GetFuncDescfunction #98300
-
-
Python/ceval_gil.h -
Python/codecs.c -
Python/compile.c- write_instr is not handling the case where ilen > 2. It might be that those are never seen in practice...? If so, feel free to close this bug.
- check_ann_subscr doesn't have any coverage for slice or tuple kinds.
- optimize_basic_block has some opcodes that aren't covered in the JUMP_IF_FALSE_OR_POP and the JUMP_IF_TRUE_OR_POP cases.
-
Python/condvar.h -
Python/context.c-
PyContext_Copy,PyContext_Enter,PyContext_Exithave no coverage
-
-
Python/deepfreeze/deepfreeze.c -
Python/dtoa.c -
Python/dup2.c -
Python/dynamic_annotations.c -
Python/errors.c -
Python/fileutils.c-
is_valid_wide_chardoesn't test error branches -
encode_ascii/decode_asciihas no coverage (probably very low priority -- comment says only for platforms with a broken mbstowcs (FreeBSD, OpenIndiana) -
_Py_stathas no coverage
-
-
Python/formatter_unicode.c -
Python/frame.c -
Python/frozenmain.c -
Python/future.c -
Python/getargs.c -
Python/getopt.c -
Python/hamt.c -
Python/hashtable.c -
Python/import.c -
Python/importdl.c -
Python/initconfig.c -
Python/marshal.c -
Python/modsupport.c -
Python/mysnprintf.c -
Python/mystrtoul.c -
Python/pathconfig.c -
Python/preconfig.c -
Python/pyarena.c -
Python/pyfpe.c -
Python/pyhash.c -
Python/pylifecycle.c -
Python/pystate.c -
Python/pystrcmp.c -
Python/pystrhex.c -
Python/pystrtod.c -
Python/pythonrun.c -
Python/pytime.c -
Python/specialize.c -
Python/structmember.c -
Python/suggestions.c -
Python/symtable.c -
Python/sysmodule.c -
Python/thread.c -
Python/traceback.c- tracebacks with angle-bracketed filenames [coverage ] Missing test for tracebacks with angle bracketed filename #95259
- tb_printinternal with
depth > limit -
_PyTraceBack_Print_Indentedwith overflowingtracebacklimit - No coverage for
_Py_DumpDecimal,_Py_DumpHexadecimal,_Py_DumpASCII,dump_frame,dump_traceback,_Py_DumpTraceback,write_thread_id,_Py_DumpTracebackThreads-- possibly they have tests which are disabled under some circumstances.