-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
DOC: add doc on ExtensionArray and extending pandas #19936
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
dcd125b
0842fcf
e1f3f60
057a4f9
51097c6
10a25cf
59f83ce
87aca05
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -262,3 +262,30 @@ Data validation | |
|
||
Engarde is a lightweight library used to explicitly state your assumptions abour your datasets | ||
and check that they're *actually* true. | ||
|
||
.. _ecosystem.extensions: | ||
|
||
Extension Data Types | ||
-------------------- | ||
|
||
`cyberpandas`_ | ||
~~~~~~~~~~~~~~ | ||
|
||
Cyberpandas provides an extension type for storing arrays of IP Addresses. These | ||
arrays can be stored inside pandas' Series and DataFrame. | ||
|
||
.. _ecosystem.accessors: | ||
------------------------ | ||
|
||
A directory of projects providing extension accessors. This is for users to | ||
discover new accessors and for libraries authors to coordinate on the namespace. | ||
|
||
============== ========== ================= | ||
Library Accessor Classes | ||
============== ========== ================= | ||
`cyberpandas`_ ``ip`` Series | ||
`pdvega`_ ``vgplot`` Series, DataFrame | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. could put double backticks around Series and DataFrame in this table |
||
============== ========== ================= | ||
|
||
.. _cyberpandas: https://cyberpandas.readthedocs.io/en/latest | ||
.. _pdvega: https://jakevdp.github.io/pdvega/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -52,18 +52,24 @@ This can be a convenient way to extend pandas objects without subclassing them. | |
If you write a custom accessor, make a pull request adding it to our | ||
:ref:`ecosystem` page. | ||
|
||
Extension Arrays | ||
---------------- | ||
.. _extending.extension-types: | ||
|
||
Extension Types | ||
--------------- | ||
|
||
Pandas defines an interface for implementing data types and arrays that *extend* | ||
NumPy's type system. Pandas uses itself for some types that aren't built into | ||
NumPy (categorical, period, interval, datetime with timezone). | ||
NumPy's type system. Pandas iteself uses the extension system for some types | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. iteself --> itself |
||
that aren't built into NumPy (categorical, period, interval, datetime with | ||
timezone). | ||
|
||
Libraries can define an custom array and data type. When pandas encounters these | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. an -> a |
||
objects, they will be handled properly (i.e. not converted to an ndarray of | ||
objects). Many methods like :func:`pandas.isna` will dispatch to the extension | ||
type's implementation. | ||
|
||
If you're building a library that implements the interface, please publicize it | ||
on :ref:`ecosystem.extensions`. | ||
|
||
The interface consists of two classes. | ||
|
||
``ExtensionDtype`` | ||
|
@@ -81,10 +87,24 @@ See ``pandas/core/dtypes/base.py`` for interface definition. | |
``ExtensionArray`` | ||
"""""""""""""""""" | ||
|
||
This is the main object. | ||
This class provides all the ar 8000 ray-like functionality. ExtensionArrays are | ||
limited to 1 dimension. An ExtensionArray is linked to an ExtensionDtype via the | ||
``dtype`` attribute. | ||
|
||
Pandas makes no restrictions on how an extension array is created via its | ||
``__new__`` or ``__init__``, and puts no restrictions on how you store your | ||
data. We do require that your array be convertible to a NumPy array, even if | ||
this is relatively expensive (as it is for ``Categorical``). | ||
|
||
They may be backed by none, one, or many NumPy ararys. For example, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ararys --> arrays |
||
``pandas.Categorical`` is an extension array backed by two arrays, | ||
one for codes and one for categories. An array of IPv6 address may | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. address --> addresses |
||
be backed by a NumPy structured array with two fields, one for the | ||
lower 64 bits and one for the upper 64 bits. Or they may be backed | ||
by some other storage type, like Python lists. | ||
|
||
See ``pandas/core/arrays/base. 8000 py`` for the interface definition. | ||
See ``pandas/core/arrays/base.py`` for the interface definition. The docstrings | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you can do a direct reference to code here (e.g. the github link) |
||
and comments contain guidance for properly implementing the interface. | ||
|
||
.. _ref-subclassing-pandas: | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
libraries -> library