8000 Categorical.(get|from)_dummies by clbarnes · Pull Request #34426 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

Categorical.(get|from)_dummies #34426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 47 commits into from
Closed

Categorical.(get|from)_dummies #34426

Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b5ab7f2
Categorical (to|from)_dummies methods
clbarnes May 27, 2020
f937c96
Tests: Categorical.(to|from)_dummies
clbarnes May 28, 2020
dd14132
Add reference to Categorical.to_dummies to get_dummies
clbarnes May 28, 2020
9dc9da5
whatsnew: add issue number to Categorical.(to|from)_dummies
clbarnes May 28, 2020
ac9cec2
Review comments for dummies tests
clbarnes May 28, 2020
0459cb1
Review comments for dummies implementation
clbarnes May 28, 2020
65e68c2
dummies review comments
clbarnes May 29, 2020
1334026
User guide: Describe Categorical.(to|from)_dummies
clbarnes May 29, 2020
c2240b6
Fix user guide errors
clbarnes May 29, 2020
66771bf
Fix numpy element from sequence error
clbarnes May 29, 2020
4e769da
Test to_dummies column type cast
clbarnes May 29, 2020
fe002af
Test review comments
clbarnes May 29, 2020
097f2c6
Review comments for implementation
clbarnes May 29, 2020
afe8eda
Fix doctest for missing values
clbarnes May 29, 2020
e78158e
xfail for Categorical from sparse
clbarnes 8000 May 29, 2020
4fb1e5e
Fix tests
clbarnes May 29, 2020
9fa5494
fix isna
clbarnes May 29, 2020
61567fd
Explicit integer type cast
clbarnes May 29, 2020
5d724cc
Test categorical <-> dummies roundtrip
clbarnes May 29, 2020
1182ce5
more type casts
clbarnes May 29, 2020
04ca72a
Add wiki link for dummy variables
clbarnes Sep 17, 2020
6e4f71a
Remove deprecated numpy <v1.16 check
clbarnes Sep 17, 2020
a761baf
isort fix
clbarnes Sep 17, 2020
ed58c77
undo changes to whatsnew v1.1.0
clbarnes Sep 17, 2020
741cf8f
whatsnew/v1.2.0: Categorical get/from dummies
clbarnes Sep 17, 2020
6f199b6
Update user_guide/categorical docs
clbarnes Sep 17, 2020
034f8e1
Reference Categorical.get_dummies in reshape.py
clbarnes Sep 17, 2020
b80f089
Categorical->dummies more like get_dummies
clbarnes Sep 17, 2020
0eb936f
categorical tests
clbarnes Sep 17, 2020
8f212e1
isort pandas_web
clbarnes Sep 17, 2020
bda5265
fix _get_dummies_1d import path
clbarnes Sep 17, 2020
6e6ddda
categorical.test_api: to->get dummies
clbarnes Sep 17, 2020
9fcebf0
isort pandas_web
clbarnes Sep 17, 2020
b9908c4
fix typos in categorical doctests
clbarnes Sep 17, 2020
faeec41
isort test_datetime
clbarnes Sep 17, 2020
e11f28e
use get_dummies instead of _get_dummies_1d
clbarnes Sep 17, 2020
742c940
Reference get_dummies/ from_dummies in reshaping docs
clbarnes Sep 17, 2020
722137d
use prefix in from_dummies
clbarnes Sep 17, 2020
4945ba8
document prefix handling in categorical.rst
clbarnes Sep 17, 2020
1f98233
Lower-memory impl for Categorical.from_dummies
clbarnes Sep 17, 2020
ff01048
remove comment about use of _get_dummies_1d
clbarnes Sep 22, 2020
604b839
type-annotate get/from_dummies
clbarnes Sep 22, 2020
c71e807
split overlong line
clbarnes Sep 22, 2020
6f9272a
blacken
clbarnes Sep 22, 2020
8fd4b72
use f-strings
clbarnes Sep 22, 2020
534bc33
add some typing
clbarnes Sep 22, 2020
0facec6
remove unnecessary .values
clbarnes Sep 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
use prefix in from_dummies
  • Loading branch information
@clbarnes
clbarnes committed Sep 22, 2020
commit 722137d374639fbd5fc14c1b12b39b92a74f1640
27 changes: 26 additions & 1 deletion pandas/core/arrays/categorical.py
View file
< 4AF4 td class="blob-code blob-code-addition js-file-line"> dummies = dummies[to_keep].rename(columns=name_map)
Original file line number Diff line number Diff line change
Expand Up @@ -375,10 +375,17 @@ def __init__(

@classmethod
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree with the others

either remove this or make it identical to get_dummies

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the discussion around this was more about the necessity/ implementation of to_dummies - there currently isn't an equivalent of from_dummies in pandas, which is what originally precipitated the issue. But note taken re. to_dummies if that was the intention.

def from_dummies(
cls, dummies: "DataFrame", ordered: Optional[bool] = None
cls,
dummies: "DataFrame",
ordered: Optional[bool] = None,
prefix=None,
prefix_sep="_",
) -> "Categorical":
"""Create a `Categorical` using a ``DataFrame`` of dummy variables.

Can use a subset of columns based on the ``prefix``
and ``prefix_sep`` parameters.

The ``DataFrame`` must have no more than one truthy value per row.
The columns of the ``DataFrame`` become the categories of the `Categorical`.
A column whose header is NA will be dropped:
Expand All @@ -391,6 +398,13 @@ def from_dummies(
Sparse dataframes are not supported.
ordered : bool
Whether or not this Categorical is ordered.
prefix : optional str
Only take columns whose names are strings starting
with this prefix and ``prefix_sep``,
stripping those elements from the resulting category names.
prefix_sep : str, default "_"
If ``prefix`` is not ``None``, use as the separator
between the prefix and the final name of the category.

Raises
------
Expand Down Expand Up @@ -433,6 +447,17 @@ def from_dummies(
to_drop = dummies.columns[isna(dummies.columns.values)]
if len(to_drop):
dummies = dummies.drop(columns=to_drop)

if prefix is not None:
pref = prefix + (prefix_sep or "")
name_map = dict()
to_keep = []
for c in dummies.columns:
if isinstance(c, str) and c.startswith(pref):
to_keep.append(c)
name_map[c] = c[len(pref) :]

df = dummies.astype("boolean")

multicat_rows = df.sum(axis=1, skipna=False) > 1
Expand Down
0