8000 PERF: ensure_string_array with non-numpy input array by topper-123 · Pull Request #37371 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

PERF: ensure_string_array with non-numpy input array #37371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 26, 2020
Merged

PERF: ensure_string_array with non-numpy input array #37371

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1cd53fa
REGR/PERF: Index.is_
topper-123 Oct 25, 2020
986f97a
Merge branch 'master' of https://github.com/pandas-dev/pandas
topper-123 Oct 26, 2020
a0c1ec0
PERF: ensure_string_array with non-numpy input array
topper-123 Oct 23, 2020
7b4928c
fix conversion of nan to string
topper-123 Oct 23, 2020
f28792d
fix conversion
topper-123 Oct 24, 2020
3262d8f
add whatsnew, ASVs
topper-123 Oct 24, 2020
d9f8e6e
is_extension_dtype -> hasattr
topper-123 Oct 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix conversion
  • Loading branch information
@topper-123
topper-123 committed Oct 26, 2020
commit f28792d0fb4e9f8ee5bfff44b273c9711a22b04c
24 changes: 14 additions & 10 deletions pandas/_libs/lib.pyx
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -651,29 +651,33 @@ cpdef ndarray[object] ensure_string_array(
cdef:
Py_ssize_t i = 0, n = len(arr)

from pandas.core.dtypes.common import is_extension_array_dtype

if is_extension_array_dtype(arr):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we avoid doing this as it circularizs things, you can check if it has a .to_numpy method or maybe @jbrockmendel has a better way here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've changed to use hasattr(arr, "to_numpy").

arr = arr.to_numpy()
elif not isinstance(arr, np.ndarray):
arr = np.array(arr, dtype=object)

result = np.asarray(arr, dtype="object")

if copy and result is arr:
result = result.copy()

arr = np.asarray(arr) # PERF: need a numpy array to ensure fast access

for i in range(n):
arr_val = arr[i]
res_val = result[i]
val = arr[i]

if not checknull(res_val) and isinstance(arr_val, str):
if isinstance(val, str):
continue

if not checknull(res_val):
result[i] = str(arr_val)
if not checknull(val):
result[i] = str(val)
else:
if convert_na_value:
arr_val = na_value
val = na_value
if skipna:
result[i] = arr_val
result[i] = val
else:
result[i] = str(arr_val)
result[i] = str(val)

return result

Expand Down
0