Data and tooling to compare the API surfaces of various array libraries.
The goal of this repository is to compare the public API surfaces of various PyData array libraries in order to better understand existing practice. In analyzing both the commonalities and differences across array libraries, we can derive a common API subset which can be standardized and used to ensure consistency (naming and otherwise) across array libraries. This API subset should include attribute names, method names, and positional and keyword arguments.
By deriving a common API subset, we can reduce friction among library consumers by reducing the cognitive overhead of learning array dialects. This is exemplified by the following user story:
As an array library author, I know that, regardless of the input array, whether NumPy, Dask, PyTorch, etc, the array has a method to compute the transpose which is guaranteed to have options
x,y, andz.
Currently, the needs of the library author in the above user story are not met, as libraries vary in their naming conventions and the optional arguments they support.
Through specification and array library compliance, we facilitate array interoperability for both users and library developers.
Currently, the following array libraries are evaluated:
- NumPy: serves as the reference API against which all other array libraries are compared.
- CuPy
- Dask.array
- JAX
- MXNet
- PyTorch
- rnumpy: an opinionated curation of NumPy APIs, serving as an exercise in evaluating what is most "essential" (i.e., the smallest set of building block functionality on which most array functionality can be built).
- PyData/Sparse
- TensorFlow
Navigate to the directory into which you want to clone this repository
$ cd ./repository/destination/directoryNext, clone the repository
$ git clone https://github.com/data-apis/array-api-comparison.gitOnce cloned, navigate to the repository directory
$ cd ./array-api-comparisonCreate an Anaconda environment
$ conda create -n array-api-comparison -c conda-forge python=3.8 nodejs jupyterlabTo activate the environment,
$ conda activate array-api-comparisonRun the installation sequence
$ makeUsage: make <cmd>
make help Print this message.
make view-docs View all array API tables.
make view-join View cross-library array API data.
make view-intersection View the intersection of array library
APIs.
make view-intersection-ranks View a table ranking the intersection
of array library APIs.
make view-common-apis View relatively common array library
APIs.
make view-common-apis-ranks View a table ranking relatively common
array library APIs.
make view-complement View array library APIs which are not
in the intersection.
make view-common-complement View array library APIs which are not
among the list of relatively common
APIs.
make view-lib-top-k-common View a table displaying the top `K`
(relatively) common array library APIs
across various libraries.
make view-lib-top-k-complement View a table displaying the top K array
library APIs in the complement across
various libraries.
make view-lib-top-k-common-complement View a table displaying the top `K`
array library APIs in the complement of
the list of (relatively) common APIs
across various libraries.
To run the Jupyter notebooks, run
$ jupyter labThis repository contains the following directories:
- data: array API data (e.g., array library APIs and their NumPy equivalents).
- docs: browser-based documentation for viewing array API data.
- etc: configuration files.
- notebooks: Jupyter notebooks for analysis.
- scripts: scripts for data manipulation and documentation generation.
- tools: project tooling.
The data directory contains the following directories
- raw: raw array library API data.
- joins: array library APIs matched to their NumPy equivalents.
- vendor: datasets acquired from third party sources, such as those found in the Python API Record repository.
The raw data directory contains the following datasets:
XXXXX.(csv|json): raw array library API data.
The joins data directory contains the following datasets:
XXXXX_numpy.(csv|json): array library APIs and their NumPy equivalents.
Lastly, the root data directory contains the following additional datasets:
join.(csv|json): array library API data combined in a single file.intersection.(csv|json): array library API intersection.common_apis.(csv|json): array library APIs which are (relatively) common across downstream libraries (>67%).complement.(csv|json): array library APIs which are not in the intersection.intersection_ranks.(csv|json): array library APIs which are in the intersection ranked according to relative usage in downstream libraries.common_apis_ranks.(csv|json): array library APIs which are in the list of (relatively) common APIs ranked according to relative usage in downstream libraries.lib_top_k_common.(csv|json): the topKarray library API names in the list of relatively common APIs per downstream library according to relative usage.lib_top_k_common_complement.(csv|json): the topKarray library API names not in the list of relatively common APIs per downstream library according to relative usage.lib_top_k_complement.(csv|json): the topKarray library API names not in the list of API intersection per downstream library according to relative usage.lib_top_100_category_stats.(csv|json): categorization statistics for the top100NumPy APIs which are consumed for each downstream library.
Note: the datasets in the root data directory are generated.
When editing data files, consider the JSON data to be the source of truth. CSV files are generated from the JSON data.
To contribute array API data to this repository, add an data/joins/XXXXX_numpy.json file, where XXXXX is the lowercase name of the relevant array library (e.g., cupy). The JSON file should include a JSON array, where each array element has the following fields:
name: array library API name.numpy: NumPy API equivalent.
For example,
[
{
"name": "all",
"numpy": "numpy.all"
},
{
"name": "allclose",
"numpy": "numpy.allclose"
},
...
]
Once added, the CSV variant can be generated using internal tooling.