8000 Categorical.(get|from)_dummies by clbarnes · Pull Request #34426 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

Categorical.(get|from)_dummies #34426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 47 commits into from
Closed

Categorical.(get|from)_dummies #34426

Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b5ab7f2
Categorical (to|from)_dummies methods
clbarnes May 27, 2020
f937c96
Tests: Categorical.(to|from)_dummies
clbarnes May 28, 2020
dd14132
Add reference to Categorical.to_dummies to get_dummies
clbarnes May 28, 2020
9dc9da5
whatsnew: add issue number to Categorical.(to|from)_dummies
clbarnes May 28, 2020
ac9cec2
Review comments for dummies tests
clbarnes May 28, 2020
0459cb1
Review comments for dummies implementation
clbarnes May 28, 2020
65e68c2
dummies review comments
clbarnes May 29, 2020
1334026
User guide: Describe Categorical.(to|from)_dummies
clbarnes May 29, 2020
c2240b6
Fix user guide errors
clbarnes May 29, 2020
66771bf
Fix numpy element from sequence error
clbarnes May 29, 2020
4e769da
Test to_dummies column type cast
clbarnes May 29, 2020
fe002af
Test review comments
clbarnes May 29, 2020
097f2c6
Review comments for implementation
clbarnes May 29, 2020
afe8eda
Fix doctest for missing values
clbarnes May 29, 2020
e78158e
xfail for Categorical from sparse
clbarnes May 29, 2020
4fb1e5e
Fix tests
clbarnes May 29, 2020
9fa5494
fix isna
clbarnes May 29, 2020
61567fd
Explicit integer type cast
clbarnes May 29, 2020
5d724cc
Test categorical <-> dummies roundtrip
clbarnes May 29, 2020
1182ce5
more type casts
clbarnes May 29, 2020
04ca72a
Add wiki link for dummy variables
clbarnes Sep 17, 2020
6e4f71a
Remove deprecated numpy <v1.16 check
clbarnes Sep 17, 2020
a761baf
isort fix
clbarnes Sep 17, 2020
ed58c77
undo changes to whatsnew v1.1.0
clbarnes Sep 17, 2020
741cf8f
whatsnew/v1.2.0: Categorical get/from dummies
clbarnes Sep 17, 2020
6f199b6
Update user_guide/categorical docs
clbarnes Sep 17, 2020
034f8e1
Reference Categorical.get_dummies in reshape.py
clbarnes Sep 17, 2020
b80f089
Categorical->dummies more like get_dummies
clbarnes Sep 17, 2020
0eb936f
categorical tests
clbarnes Sep 17, 2020
8f212e1
isort pandas_web
clbarnes Sep 17, 2020
bda5265
fix _get_dummies_1d import path
clbarnes Sep 17, 2020
6e6ddda
categorical.test_api: to->get dummies
clbarnes Sep 17, 2020
9fcebf0
isort pandas_web
clbarnes Sep 17, 2020
b9908c4
fix typos in categorical doctests
clbarnes Sep 17, 2020
faeec41
isort test_datetime
clbarnes Sep 17, 2020
e11f28e
use get_dummies instead of _get_dummies_1d
clbarnes Sep 17, 2020
742c940
Reference get_dummies/ from_dummies in reshaping docs
clbarnes Sep 17, 2020
722137d
use prefix in from_dummies
clbarnes Sep 17, 2020
4945ba8
document prefix handling in categorical.rst
clbarnes Sep 17, 2020
1f98233
Lower-memory impl for Categorical.from_dummies
clbarnes Sep 17, 2020
ff01048
remove comment about use of _get_dummies_1d
clbarnes Sep 22, 2020
604b839
type-annotate get/from_dummies
clbarnes Sep 22, 2020
c71e807
split overlong line
clbarnes Sep 22, 2020
6f9272a
blacken
clbarnes Sep 22, 2020
8fd4b72
use f-strings
clbarnes Sep 22, 2020
534bc33
add some typing
clbarnes Sep 22, 2020
0facec6
remove unnecessary .values
clbarnes Sep 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
blacken
  • Loading branch information
@clbarnes
clbarnes committed Sep 22, 2020
commit 6f9272a3722577da8df11e940bab8034b05dec86
7 changes: 3 additions & 4 deletions pandas/core/arrays/categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,11 +451,9 @@ def from_dummies(
"""
from pandas import Series

copied = False
to_drop = dummies.columns[isna(dummies.columns.values)]
if len(to_drop):
dummies = dummies.drop(columns=to_drop)
copied = True

if prefix is None:
cats = dummies.columns
Expand All @@ -471,7 +469,7 @@ def from_dummies(

df = dummies.astype("boolean")
if fillna is not None:
df = df.fillna(fillna, inplace=copied)
df = df.fillna(fillna)

row_totals = df.sum(axis=1, skipna=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why skipna?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is no explicit fillna policy given, and there are still NA values in the data, I'd prefer to raise an error rather than silently pass bunk data through. Therefore nans should not be skipped in this step so that they can be checked for in the next line.

if row_totals.isna().any():
Expand All @@ -481,7 +479,8 @@ def from_dummies(
if multicat_rows.any():
raise ValueError(
"{} record(s) belongs to multiple categories: {}".format(
multicat_rows.sum(), list(df.index[multicat_rows]),
multicat_rows.sum(),
list(df.index[multicat_rows]),
)
)

Expand Down
0