8000 ENH: add consortium standard entrypoint (#54383) · mroeschke/pandas@04943ab · GitHub
[go: up one dir, main page]
Skip to content

Commit 04943ab

Browse files
MarcoGorellimroeschke
MarcoGorelli
authored and
mroeschke
committed
ENH: add consortium standard entrypoint (pandas-dev#54383)
* add consortium standard entrypoint * add dataframe-api-compat to optional dependencies, add to test_downstream * align * use importorskip * mypy * fixup table in docs * whatsnew * add check to package-checks.yml
1 parent 8b3dee1 commit 04943ab

File tree

12 files changed

+134
-60
lines changed

12 files changed

+134
-60
lines changed

‎.github/workflows/package-checks.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
runs-on: ubuntu-22.04
2525
strategy:
2626
matrix:
27-
extra: ["test", "performance", "computation", "fss", "aws", "gcp", "excel", "parquet", "feather", "hdf5", "spss", "postgresql", "mysql", "sql-other", "html", "xml", "plot", "output_formatting", "clipboard", "compression", "all"]
27+
extra: ["test", "performance", "computation", "fss", "aws", "gcp", "excel", "parquet", "feather", "hdf5", "spss", "postgresql", "mysql", "sql-other", "html", "xml", "plot", "output_formatting", "clipboard", "compression", "consortium-standard", "all"]
2828
fail-fast: false
2929
name: Install Extras - ${{ matrix.extra }}
3030
concurrency:

‎ci/deps/actions-311-downstream_compat.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,5 +73,6 @@ dependencies:
7373
- pyyaml
7474
- py
7575
- pip:
76+
- dataframe-api-compat>=0.1.7
7677
- pyqt5>=5.15.6
7778
- tzdata>=2022.1

‎ci/deps/actions-39-minimum_versions.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,5 +61,6 @@ dependencies:
6161
- zstandard=0.17.0
6262

6363
- pip:
64+
- dataframe-api-compat==0.1.7
6465
- pyqt5==5.15.6
6566
- tzdata==2022.1

‎doc/source/getting_started/install.rst‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,3 +415,14 @@ brotli 0.7.0 compression Brotli compression
415415
python-snappy 0.6.1 compression Snappy compression
416416
Zstandard 0.17.0 compression Zstandard compression
417417
========================= ================== =============== =============================================================
418+
419+
Consortium Standard
420+
^^^^^^^^^^^^^^^^^^^
421+
422+
Installable with ``pip install "pandas[consortium-standard]"``
423+
424+
========================= ================== =================== =============================================================
425+
Dependency Minimum Version pip extra Notes
426+
========================= ================== =================== =============================================================
427+
dataframe-api-compat 0.1.7 consortium-standard Consortium Standard-compatible implementation based on pandas
428+
========================= ================== =================== =============================================================

‎doc/source/whatsnew/v2.1.0.rst‎

Lines changed: 62 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,7 @@ Other enhancements
218218
- Many read/to_* functions, such as :meth:`DataFrame.to_pickle` and :func:`read_csv`, support forwarding compression arguments to lzma.LZMAFile (:issue:`52979`)
219219
- Reductions :meth:`Series.argmax`, :meth:`Series.argmin`, :meth:`Series.idxmax`, :meth:`Series.idxmin`, :meth:`Index.argmax`, :meth:`Index.argmin`, :meth:`DataFrame.idxmax`, :meth:`DataFrame.idxmin` are now supported for object-dtype objects (:issue:`4279`, :issue:`18021`, :issue:`40685`, :issue:`43697`)
220220
- :meth:`DataFrame.to_parquet` and :func:`read_parquet` will now write and read ``attrs`` respectively (:issue:`54346`)
221+
- Added support for the DataFrame Consortium Standard (:issue:`54383`)
221222
- Performance improvement in :meth:`GroupBy.quantile` (:issue:`51722`)
222223

223224
.. ---------------------------------------------------------------------------
@@ -256,65 +257,67 @@ Increased minimum versions for dependencies
256257
Some minimum supported versions of dependencies were updated.
257258
If installed, we now require:
258259

259-
+-----------------+-----------------+----------+---------+
260-
| Package | Minimum Version | Required | Changed |
261-
+=================+=================+==========+=========+
262-
| numpy | 1.22.4 | X | X |
263-
+-----------------+-----------------+----------+---------+
264-
| mypy (dev) | 1.4.1 | | X |
265-
+-----------------+-----------------+----------+---------+
266-
| beautifulsoup4 | 4.11.1 | | X |
267-
+-----------------+-----------------+----------+---------+
268-
| bottleneck | 1.3.4 | | X |
269-
+-----------------+-----------------+----------+---------+
270-
| fastparquet | 0.8.1 | | X |
271-
+-----------------+-----------------+----------+---------+
272-
| fsspec | 2022.05.0 | | X |
273-
+-----------------+-----------------+----------+---------+
274-
| hypothesis | 6.46.1 | | X |
275-
+-----------------+-----------------+----------+---------+
276-
| gcsfs | 2022.05.0 | | X |
277-
+-----------------+-----------------+----------+---------+
278-
| jinja2 | 3.1.2 | | X |
279-
+-----------------+-----------------+----------+---------+
280-
| lxml | 4.8.0 | | X |
281-
+-----------------+-----------------+----------+---------+
282-
| numba | 0.55.2 | | X |
283-
+-----------------+-----------------+----------+---------+
284-
| numexpr | 2.8.0 | | X |
285-
+-----------------+-----------------+----------+---------+
286-
| openpyxl | 3.0.10 | | X |
287-
+-----------------+-----------------+----------+---------+
288-
| pandas-gbq | 0.17.5 | | X |
289-
+-----------------+-----------------+----------+---------+
290-
| psycopg2 | 2.9.3 | | X |
291-
+-----------------+-----------------+----------+---------+
292-
| pyreadstat | 1.1.5 | | X |
293-
+-----------------+-----------------+----------+---------+
294-
| pyqt5 | 5.15.6 | | X |
295-
+-----------------+-----------------+----------+---------+
296-
| pytables | 3.7.0 | | X |
297-
+-----------------+-----------------+----------+---------+
298-
| pytest | 7.3.2 | | X |
299-
+-----------------+-----------------+----------+---------+
300-
| python-snappy | 0.6.1 | | X |
301-
+-----------------+-----------------+----------+---------+
302-
| pyxlsb | 1.0.9 | | X |
303-
+-----------------+-----------------+----------+---------+
304-
| s3fs | 2022.05.0 | | X |
305-
+-----------------+-----------------+----------+---------+
306-
| scipy | 1.8.1 | | X |
307-
+-----------------+-----------------+----------+---------+
308-
| sqlalchemy | 1.4.36 | | X |
309-
+-----------------+-----------------+----------+---------+
310-
| tabulate | 0.8.10 | | X |
311-
+-----------------+-----------------+----------+---------+
312-
| xarray | 2022.03.0 | | X |
313-
+-----------------+-----------------+----------+---------+
314-
| xlsxwriter | 3.0.3 | | X |
315-
+-----------------+-----------------+----------+---------+
316-
| zstandard | 0.17.0 | | X |
317-
+-----------------+-----------------+----------+---------+
260+
+----------------------+-----------------+----------+---------+
261+
| Package | Minimum Version | Required | Changed |
262+
+======================+=================+==========+=========+
263+
| numpy | 1.22.4 | X | X |
264+
+----------------------+-----------------+----------+---------+
265+
| mypy (dev) | 1.4.1 | | X |
266+
+----------------------+-----------------+----------+---------+
267+
| beautifulsoup4 | 4.11.1 | | X |
268+
+----------------------+-----------------+----------+---------+
269+
| bottleneck | 1.3.4 | | X |
270+
+----------------------+-----------------+----------+---------+
271+
| dataframe-api-compat | 0.1.7 | | X |
272+
+----------------------+-----------------+----------+---------+
273+
| fastparquet | 0.8.1 | | X |
274+
+----------------------+-----------------+----------+---------+
275+
| fsspec | 2022.05.0 | | X |
276+
+----------------------+-----------------+----------+---------+
277+
| hypothesis | 6.46.1 | | X |
278+
+----------------------+-----------------+----------+---------+
279+
| gcsfs | 2022.05.0 | | X |
280+
+----------------------+-----------------+----------+---------+
281+
| jinja2 | 3.1.2 | | X |
282+
+----------------------+-----------------+----------+---------+
283+
| lxml | 4.8.0 | | X |
284+
+----------------------+-----------------+----------+---------+
285+
| numba | 0.55.2 | | X |
286+
+----------------------+-----------------+----------+---------+
287+
| numexpr | 2.8.0 | | X |
288+
+----------------------+-----------------+----------+---------+
289+
| openpyxl | 3.0.10 | | X |
290+
+----------------------+-----------------+----------+---------+
291+
| pandas-gbq | 0.17.5 | | X |
292+
+----------------------+-----------------+----------+---------+
293+
| psycopg2 | 2.9.3 | | X |
294+
+----------------------+-----------------+----------+---------+
295+
| pyreadstat | 1.1.5 | | X |
296+
+----------------------+-----------------+----------+---------+
297+
| pyqt5 | 5.15.6 | | X |
298+
+----------------------+-----------------+----------+---------+
299+
| pytables | 3.7.0 | | X |
300+
+----------------------+-----------------+----------+---------+
301+
| pytest | 7.3.2 | | X |
302+
+----------------------+-----------------+----------+---------+
303+
| python-snappy | 0.6.1 | | X |
304+
+----------------------+-----------------+----------+---------+
305+
| pyxlsb | 1.0.9 | | X |
306+
+----------------------+-----------------+----------+---------+
307+
| s3fs | 2022.05.0 | | X |
308+
+----------------------+-----------------+----------+---------+
309+
| scipy | 1.8.1 | | X |
310+
+----------------------+-----------------+----------+---------+
311+
| sqlalchemy | 1.4.36 | | X |
312+
+----------------------+-----------------+----------+---------+
313+
| tabulate | 0.8.10 | | X |
314+
+----------------------+-----------------+----------+---------+
315+
| xarray | 2022.03.0 | | X |
316+
+----------------------+-----------------+----------+---------+
317+
| xlsxwriter | 3.0.3 | | X |
318+
+----------------------+-----------------+----------+---------+
319+
| zstandard | 0.17.0 | | X |
320+
+----------------------+-----------------+----------+---------+
318321

319322
For `optional libraries <https://pandas.pydata.org/docs/getting_started/install.html>`_ the general recommendation is to use the latest version.
320323
The following table lists the lowest version per library that is currently being tested throughout the development of pandas.

‎environment.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ dependencies:
115115
- pygments # Code highlighting
116116

117117
- pip:
118+
- dataframe-api-compat>=0.1.7
118119
- sphinx-toggleprompt # conda-forge version has stricter pins on jinja2
119120
- typing_extensions; python_version<"3.11"
120121
- tzdata>=2022.1

‎pandas/compat/_optional.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
"blosc": "1.21.0",
2020
"bottleneck": "1.3.4",
2121
"brotli": "0.7.0",
22+
"dataframe-api-compat": "0.1.7",
2223
"fastparquet": "0.8.1",
2324
"fsspec": "2022.05.0",
2425
"html5lib": "1.1",

‎pandas/core/frame.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -932,6 +932,21 @@ def __dataframe__(
932932

933933
return PandasDataFrameXchg(self, nan_as_null, allow_copy)
934934

935+
def __dataframe_consortium_standard__(
936+
self, *, api_version: str | None = None
937+
) -> Any:
938+
"""
939+
Provide entry point to the Consortium DataFrame Standard API.
940+
941+
This is developed and maintained outside of pandas.
942+
Please report any issues to https://github.com/data-apis/dataframe-api-compat.
943+
"""
944+
dataframe_api_compat = import_optional_dependency("dataframe_api_compat")
945+
convert_to_standard_compliant_dataframe = (
946+
dataframe_api_compat.pandas_standard.convert_to_standard_compliant_dataframe
947+
)
948+
return convert_to_standard_compliant_dataframe(self, api_version=api_version)
949+
935950
# ----------------------------------------------------------------------
936951

937952
@property

‎pandas/core/series.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
from pandas._libs.lib import is_range_indexer
4040
from pandas.compat import PYPY
4141
from pandas.compat._constants import REF_COUNT
42+
from pandas.compat._optional import import_optional_dependency
4243
from pandas.compat.numpy import function as nv
4344
from pandas.errors import (
4445
ChainedAssignmentError,
@@ -955,6 +956,22 @@ def __array__(self, dtype: npt.DTypeLike | None = None) -> np.ndarray:
955956
arr.flags.writeable = False
956957
return arr
957958

959+
# ----------------------------------------------------------------------
960+
961+
def __column_consortium_standard__(self, *, api_version: str | None = None) -> Any:
962+
"""
963+
Provide entry point to the Consortium DataFrame Standard API.
964+
965+
This is developed and maintained outside of pandas.
966+
Please report any issues to https://github.com/data-apis/dataframe-api-compat.
967+
"""
968+
dataframe_api_compat = import_optional_dependency("dataframe_api_compat")
969+
return (
970+
dataframe_api_compat.pandas_standard.convert_to_standard_compliant_column(
971+
self, api_version=api_version
972+
)
973+
)
974+
958975
# ----------------------------------------------------------------------
959976
# Unary Methods
960977

‎pandas/tests/test_downstream.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,6 +334,27 @@ def test_from_obscure_array(dtype, array_likes):
334334
tm.assert_index_equal(result, expected)
335335

336336

337+
def test_dataframe_consortium() -> None:
338+
"""
339+
Test some basic methods of the dataframe consortium standard.
340+
341+
Full testing is done at https://github.com/data-apis/dataframe-api-compat,
342+
this is just to check that the entry point works as expected.
343+
"""
344+
pytest.importorskip("dataframe_api_compat")
345+
df_pd = DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
346+
df = df_pd.__dataframe_consortium_standard__()
347+
result_1 = df.get_column_names()
348+
expected_1 = ["a", "b"]
349+
assert result_1 == expected_1
350+
351+
ser = Series([1, 2, 3])
352+
col = ser.__column_consortium_standard__()
353+
result_2 = col.get_value(1)
354+
expected_2 = 2
355+
assert result_2 == expected_2
356+
357+
337358
def test_xarray_coerce_unit():
338359
# GH44053
339360
xr = pytest.importorskip("xarray")

0 commit comments

Comments
 (0)
0