8000 TST Fix openml parser implementation for pandas-dev by thomasjpfan · Pull Request #26386 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

TST Fix openml parser implementation for pandas-dev #26386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 23, 2023
Merged

TST Fix openml parser implementation for pandas-dev #26386

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f456216
TST Fix openml parser implementation for pandas-dev
thomasjpfan May 16, 2023
a54cab0
DOC Adds whats new
thomasjpfan May 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion doc/whats_new/v1.3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,9 @@ Changelog
is deprecated and will be removed in v1.5.
:pr:`25784` by :user:`Jérémie du Boisberranger`.

- |Fix| :func:`datasets.fetch_openml` returns improved data types when
`as_frame=True` and `parser="liac-arff"`. :pr:`26386` by `Thomas Fan`_.

:mod:`sklearn.decomposition`
............................

Expand Down Expand Up @@ -409,7 +412,7 @@ Changelog

- |API| The `eps` parameter of the :func:`log_loss` has been deprecated and will be
removed in 1.5. :pr:`25299` by :user:`Omar Salman <OmarManzoor>`.

- |Feature| :func:`metrics.average_precision_score` now supports the
multiclass case.
:pr:`17388` by :user:`Geoffrey Bolmier <gbolmier>` and
Expand Down
5 changes: 5 additions & 0 deletions sklearn/datasets/_arff_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,11 @@ def _io_to_generator(gzip_file):
dfs.append(
pd.DataFrame(data, columns=columns_names, copy=False)[columns_to_keep]
)
# dfs[0] contains only one row, which may not have enough data to infer to
# column's dtype. Here we use `dfs[1]` to configure the dtype in dfs[0]
if len(dfs) >= 2:
dfs[0] = dfs[0].astype(dfs[1].dtypes)

frame = pd.concat(dfs, ignore_index=True)
del dfs, first_df

Expand Down
2 changes: 1 addition & 1 deletion sklearn/datasets/tests/test_openml.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -925,7 +925,7 @@ def datasets_missing_values():
# with casting it will be transformed to either float or Int64
(40966, "pandas", 1, 77, 0),
# titanic
(40945, "liac-arff", 3, 5, 0),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused by this PR: why would fixing the pandas parser impact a test for the liac-arff parser?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is fixing the liac-arff parser which uses pandas when as_frame=True.

For this specific tests, the "body" feature is interpreted as a object dtype on main. With this PR, the "body" feature is a float dtype, which increases the expected_n_floats by one.

(40945, "liac-arff", 3, 6, 0),
(40945, "pandas", 3, 3, 3),
],
)
Expand Down
0