DOC: Add ref docs for C generic types. #11689
Conversation
doc/source/reference/c-api.dtype.rst
Outdated
| Generic Typing and Deviations from C Standard Syntax | ||
| ---------------------------------------------------- | ||
|
|
||
| Many of the C source files in NumPy are not compliant with standard |
There was a problem hiding this comment.
would be good to make clear that this is about .c.src files. The current sentence suggest that we have .c files that aren't C89/99 compliant.
d1810f2 to
c0c3287
Compare
doc/source/reference/c-api.dtype.rst
Outdated
| enumerate the intended type substitutions. | ||
|
|
||
| The preprocessing of generically typed NumPy C source files is performed | ||
| by a set of Python modules (:file:`numpy/core/code_generators`). The type |
There was a problem hiding this comment.
The macros you're trying to document aren't expanded by anything in the numpy/core/code_generators directory. The code you're looking for is part of numpy-distutils in numpy/distutils/conv_template.py.
I think the "C standard compliant" wording here is a little negative about this. Numpy uses a custom code generator that it also ships. This isn't really all that crazy of a thing to do. Its not like doing this with C macros or just accepting that massive code duplication are either good choices.
There was a problem hiding this comment.
Interesting, and I see there are more "rules" than I thought in numpy/distutils/conv_template.py. I'll try to rephrase to make it a bit less framed around C standard compliance deviations, though this should likely be mentioned at least somewhere for clarity.
There was a problem hiding this comment.
Instead of "are not compliant with standard C syntax" consider something like "are written in a custom templating language that is used to generate C code"
|
The documentation might find a better home in |
c0c3287 to
18ce475
Compare
|
Revised & expanded a bit based on initial feedback; I suppose this means that any project using numpy-distutils could theoretically use the type substitution macros. If people feel that this piece of doc is more suitably placed elsewhere as Matti suggests perhaps they can chime in--I've not visited the page he suggests before so perhaps not entirely convinced on that git workflow page being the right home. |
doc/source/reference/c-api.dtype.rst
Outdated
| a substitution mechansim for the large number of data types described above. | ||
| For example, a loop in the NumPy C source code may have a :c:data:`@TYPE@` | ||
| variable which is preprocessed to a number of otherwise identical loops | ||
| with several types such as :c:data:`INT, LONG, UINT, ULONG`. |
There was a problem hiding this comment.
Those are not really type variables, they are intended as part of function names. Then actual types are strings like npy_byte. It is probably better to describe the operation as string substitution.
|
It would be good to add an example, and maybe link to the module documentation in |
|
Maybe in the |
18ce475 to
558412e
Compare
|
I've substantially revised this DOC PR in the following ways, based on feedback above and now having some experience with our template language:
Other notes / thoughts here:
|
I'm too lazy to do that, but I think you're right - especially if this is something that we're distributing as part of distutils, and not just using internally. |
558412e to
4d9aceb
Compare
|
Ok, I revised again to migrate the changes to disutils docs and adjust wording accordingly. However, this raises other issues:
|
In #12250 I linked the guide into the rendered on-line documents |
| * TIMEDELTA# | ||
| * #totype = npy_byte, npy_ubyte, npy_short, npy_ushort, npy_int, npy_uint, | ||
| * npy_long, npy_ulong, npy_longlong, npy_ulonglong, | ||
| * npy_datetime, npy_timedelta# |
There was a problem hiding this comment.
It would be good to note that:
- These sequences are combined essentially via
zip, and iterated together rather than as a cartesian product - There is a
*Nsyntax for repeated elements
| XXX: Describe how files with extensions ``.f.src``, ``.pyf.src``, | ||
| ``.c.src``, etc. are pre-processed by the ``build_src`` command. | ||
| NumPy Distutils preprocesses C source files (extension: :file:`.c.src`) written | ||
| in a custom templating language to generate C code. The :c:data:`@` symbol is |
There was a problem hiding this comment.
Only C files? It looks like it might support other files too.
|
Looking again, it seems there's a lot of overlap here with https://github.com/numpy/numpy/blob/master/doc/source/reference/distutils.rst#conversion-of-src-files. Can you try to merge these? Perhaps |
|
@eric-wieser this PR has an approval but then a request for refactoring. Is it still approved? |
|
Let's just put this in - I've opened a new issue about merging these docs |
I tried drafting a bit of reference documentation about the
@TYPE@style generic typing used in NumPy C source since even an experienced C programmer may not have seen that before (it is not standard compliant). I don't think we spell this out anywhere in docs based on a brief discussion with Matti.Also, because I may learn something about NumPy source from the criticism of this documentation.