8000 restrict columns to read for pandas.read_parquet by hoffmann · Pull Request #18155 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

restrict columns to read for pandas.read_parquet #18155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Nov 8, 2017
Merged

restrict columns to read for pandas.read_parquet #18155

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8c247c2
implement to read only columns from parquet file
hoffmann Nov 7, 2017
d00d222
fix flake8
hoffmann Nov 7, 2017
c1449f5
reference issue in tests, clarify default in docstring
hoffmann Nov 7, 2017
22663e8
fix pep8
hoffmann Nov 7, 2017
f31e6a2
add whatsnew entry
hoffmann Nov 7, 2017
f91f5f8
add feature to version v0.21.1
hoffmann Nov 7, 2017
21c5f5e
add documentation how to read columns from parquet file
hoffmann Nov 7, 2017
ef30f39
use keyword argument to pass columns
hoffmann Nov 8, 2017
54fc1c9
use check_round_tip in tests
hoffmann Nov 8, 2017
e5336b6
pep8
hoffmann Nov 8, 2017
d6baa9d
add newline before versionadded
hoffmann Nov 8, 2017
7f6e7f6
fix lint problem
hoffmann Nov 8, 2017
4b22c88
fix lint error
hoffmann Nov 8, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add documentation how to read columns from parquet file
  • Loading branch information
@hoffmann
hoffmann committed Nov 7, 2017
commit 21c5f5ea3528e451ec2653e19de094d25cdc0405
10 changes: 10 additions & 0 deletions doc/source/io.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4538,6 +4538,16 @@ Read from a parquet file.

result.dtypes

Read only certain columns of a parquet file.

.. ipython:: python
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in next PR, can you add a version added tag here (for 0.21.1)


result = pd.read_parquet('example_pa.parquet', engine='pyarrow', columns=['a', 'b'])
result = pd.read_parquet('example_fp.parquet', engine='fastparquet', columns=['a', 'b'])

result.dtypes


.. ipython:: python
:suppress:

Expand Down
0