8000 REF/BUG/API: factorizing categorical data by TomAugspurger · Pull Request #19938 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

REF/BUG/API: factorizing categorical data #19938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 15, 2018
Merged

REF/BUG/API: factorizing categorical data #19938

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
18de376
REF/BUG/API: factorizing categorical data
TomAugspurger Feb 23, 2018
9ef5be2
Explicit dtype for expected
TomAugspurger Feb 28, 2018
5e52b6f
Clean : imports / remove sort
TomAugspurger Mar 6, 2018
e19ae86
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 6, 2018
121b682
Restore sort
TomAugspurger Mar 6, 2018
a6bc405
REF: remove sort from Categorical.factorize
TomAugspurger Mar 9, 2018
0bfbc47
Updated comment
TomAugspurger Mar 9, 2018
b25f383
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 12, 2018
2688c4f
Fixed new sort algo
TomAugspurger Mar 12, 2018
2395a99
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 12, 2018
ab4f01c
Implement interface
TomAugspurger Mar 13, 2018
65150f4
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 13, 2018
ad8173b
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 13, 2018
1e006d1
Merge remote-tracking branch 'upstream/master' into categorical-facto…
TomAugspurger Mar 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixed new sort algo
  • Loading branch information
@TomAugspurger
TomAugspurger committed Mar 12, 2018
commit 2688c4f39d4ffdb40ddc1ecb6471fb11627cac7e
3 changes: 2 additions & 1 deletion pandas/core/algorithms.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,8 @@ def factorize(values, sort=False, order=None, na_sentinel=-1, size_hint=None):
from pandas.core.sorting import safe_sort
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could move to the except (but no big deal)

try:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed all t 643D he sorting from Categorical.factorize. All that logic is here.

I don't think we want to just call safe_sort for two reasons

  1. that function does a lot of unnescessary work when we know that uniques is an ndarray or EA.
  2. It coerces categoricals to object ndarrays.
  3. EAs (like Categorical) may have special sorting rules.

On some small bencharks (10,000 elements) this is about 25-40% faster. The only slow case, for which we still need safe_sort, is when the array is mixed. In that case things are about 10% slower.

order = uniques.argsort()
labels = take_1d(order, labels, fill_value=na_sentinel)
order2 = order.argsort()
labels = take_1d(order2, labels, fill_value=na_sentinel)
uniques = uniques.take(order)
except TypeError:
# Mixed types, where uniques.argsort fails.
Expand Down
0