pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with "relational" or "labeled" data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way toward this goal.
Here are just a few of the things that pandas does well:
- Easy handling of missing data (represented as
NaN) in floating point as well as non-floating point data - Size mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects
- Automatic and explicit data alignment: objects can
be explicitly aligned to a set of labels, or the user can simply
ignore the labels and let
Series,DataFrame, etc. automatically align the data for you in computations - Powerful, flexible group by functionality to perform split-apply-combine operations on data sets, for both aggregating and transforming data
- Make it easy to convert ragged, differently-indexed data in other Python and NumPy data structures into DataFrame objects
- Intelligent label-based slicing, fancy indexing, and subsetting of large data sets
- Intuitive merging and joining data sets
- Flexible reshaping and pivoting of data sets
- Hierarchical labeling of axes (possible to have multiple labels per tick)
- Robust IO tools for loading data from flat files (CSV and delimited), Excel files, databases, and saving/loading data from the ultrafast HDF5 format
- Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging, etc.
The source code is currently hosted on GitHub at: http://github.com/pydata/pandas
Binary installers for the latest released version are available at the Python package index
http://pypi.python.org/pypi/pandas/
And via easy_install:
easy_install pandasor pip:
pip install pandas- NumPy: 1.7.0 or higher
- python-dateutil: 1.5 or higher
- pytz
- Needed for time zone support with
pandas.date_range
- Needed for time zone support with
- numexpr
- Needed to accelerate some expression evaluation operations
- Required by PyTables
- bottleneck
- Needed to accelerate certain numerical operations
- Cython: Only necessary to build development version. Version 0.17.1 or higher.
- SciPy: miscellaneous statistical functions
- PyTables: necessary for HDF5-based storage
- SQLAlchemy: for SQL database support. Version 0.8.1 or higher recommended.
- matplotlib: for plotting
- statsmodels
- Needed for parts of
pandas.stats
- Needed for parts of
- For Excel I/O:
- xlrd/xlwt
- Excel reading (xlrd) and writing (xlwt)
- openpyxl
- openpyxl version 1.6.1 or higher, but lower than 2.0.0, for writing .xlsx files
- xlrd >= 0.9.0
- XlsxWriter
- Alternative Excel writer.
- xlrd/xlwt
- Google bq Command Line Tool
- Needed for
pandas.io.gbq
- Needed for
- boto: necessary for Amazon S3 access.
- One of the following combinations of libraries is needed to use the
top-level
pandas.read_htmlfunction:- BeautifulSoup4 and html5lib (Any recent version of html5lib is okay.)
- BeautifulSoup4 and lxml
- BeautifulSoup4 and html5lib and lxml
- Only lxml, although see HTML reading gotchas for reasons as to why you should probably not take this approach.
-
If you install BeautifulSoup4 you must install either lxml or html5lib or both.
pandas.read_htmlwill not work with onlyBeautifulSoup4installed. -
You are strongly encouraged to read HTML reading gotchas. It explains issues surrounding the installation and usage of the above three libraries.
-
You may need to install an older version of BeautifulSoup4:
- Versions 4.2.1, 4.1.3 and 4.0.2 have been confirmed for 64 and 32-bit Ubuntu/Debian
-
Additionally, if you're using Anaconda you should definitely read the gotchas about HTML parsing libraries
-
If you're on a system with
apt-getyou can dosudo apt-get build-dep python-lxml
to get the necessary dependencies for installation of lxml. This will prevent further headaches down the line.
To install pandas from source you need Cython in addition to the normal dependencies above. Cython can be installed from pypi:
pip install cythonIn the pandas directory (same one where you found this file after
cloning the git repo), execute:
python setup.py installor for installing in development mode:
python setup.py developAlternatively, you can use pip if you want all the dependencies pulled
in automatically (the -e option is for installing it in development
mode):
pip install -e .On Windows, you will need to install MinGW and execute:
python setup.py build --compiler=mingw32
python setup.py installSee http://pandas.pydata.org/ for more information.
BSD
The official documentation is hosted on PyData.org: http://pandas.pydata.org/
The Sphinx documentation should provide a good starting point for learning how to use the library. Expect the docs to continue to expand as time goes on.
Work on pandas started at AQR (a quantitative hedge fund) in 2008 and
has been under active development since then.
Since pandas development is related to a number of other scientific Python projects, questions are welcome on the scipy-user mailing list. Specialized discussions or design issues should take place on the PyData mailing list / Google group: