8000 feat: removed pkg_resources from all test files and moved importlib into pandas extra by kiraksi · Pull Request #1726 · googleapis/python-bigquery · GitHub
[go: up one dir, main page]
Skip to content

feat: removed pkg_resources from all test files and moved importlib into pandas extra #1726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 22 commits into from
Nov 28, 2023
Merged

feat: removed pkg_resources from all test files and moved importlib into pandas extra #1726

merged 22 commits into from
Nov 28, 2023

Conversation

kiraksi
Copy link
Contributor
@kiraksi kiraksi commented Nov 16, 2023 •
edited by parthea
Loading

In this PR

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #1725 🦕
Fixes #313 🦕

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery API. labels Nov 16, 2023
@kiraksi kiraksi added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Nov 16, 2023
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Nov 16, 2023
@kiraksi kiraksi requested a review from parthea November 16, 2023 23:29
@kiraksi kiraksi marked this pull request as ready for review November 16, 2023 23:29
@kiraksi kiraksi requested review from a team as code owners November 16, 2023 23:29
@kiraksi kiraksi requested review from mrfaizal and removed request for mrfaizal November 16, 2023 23:29
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Nov 16, 2023
parthea
parthea reviewed Nov 17, 2023
View reviewed changes
@kiraksi kiraksi requested a review from a team as a code owner November 20, 2023 22:41
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: m Pull request size is medium. labels Nov 20, 2023
@snippet-bot Snippet Bot
Copy link
snippet-bot bot commented Nov 20, 2023 •
edited
Loading

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@kiraksi kiraksi force-pushed the remove-pkgresources branch from 4cb86ab to 43e89a6 Compare November 20, 2023 22:47
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: xl Pull request size is extra large. labels Nov 20, 2023
parthea
parthea approved these changes Nov 21, 2023
View reviewed changes
Copy link
Contributor
@parthea parthea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Added one observation for potentially unnecessary version checks in test files.

@parthea parthea mentioned this pull request Nov 21, 2023
Closed
@kiraksi kiraksi added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Nov 27, 2023
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Nov 27, 2023
chalmerlowe
chalmerlowe approved these changes Nov 28, 2023
View reviewed changes
Copy link
Collaborator
@chalmerlowe chalmerlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tests/system/test_pandas.py
@pytest.mark.skipif(
PANDAS_INSTALLED_VERSION >= pkg_resources.parse_version("2.0.0"), reason=""
)
@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has several elements in the list so it is a bit easier to grok what is happening here.

@chalmerlowe chalmerlowe added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Nov 28, 2023
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Nov 28, 2023
parthea
parthea reviewed Nov 28, 2023
View reviewed changes
@kiraksi kiraksi force-pushed the remove-pkgresources branch from 0e8af65 to c50eca3 Compare November 28, 2023 21:07
@kiraksi kiraksi merged commit 1f4ebb1 into googleapis:main Nov 28, 2023
@kiraksi kiraksi deleted the remove-pkgresources branch November 28, 2023 21:47
@release-please release-please bot mentioned this pull request Nov 22, 2023
Merged
@tswast
Copy link
Contributor
tswast commented Nov 28, 2023

This breaks BigQuery DataFrames e2e tests. Let's revert.

ImportError while importing test module '/tmpfs/src/github/python-bigquery-dataframes/tests/unit/ml/test_golden_sql.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/[usr/local/lib/python3.11/importlib/__init__.py:126](https://cs.corp.google.com/piper///depot/google3/usr/local/lib/python3.11/importlib/__init__.py?l=126): in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
[tests/unit/ml/test_golden_sql.py:17](https://cs.corp.google.com/piper///depot/google3/tests/unit/ml/test_golden_sql.py?l=17): in <module>
    from google.cloud import bigquery
E   ImportError: cannot import name 'bigquery' from 'google.cloud' (/tmpfs/src/github/python-bigquery-dataframes/.nox/unit_prerelease/src/google-cloud-bigquery-storage/google/cloud/__init__.py)

https://fusion2.corp.google.com/invocations/2be1e8d5-99f1-41aa-97e4-4cabb94bd441/targets/bigframes%2Fpresubmit%2Fe2e/log

tswast added a commit that referenced this pull request Nov 28, 2023
@tswast
Revert "feat: removed pkg_resources from all test files and moved imp…
c2c5413 
…ortlib into pandas extra (#1726)"

This reverts commit 1f4ebb1.
@tswast tswast mentioned this pull request Nov 28, 2023
Closed
@parthea
Copy link
Contributor
parthea commented Nov 28, 2023 •
edited
Loading

Good catch @tswast ! I can produce the issue when using an editable install using egg as it appears in the build log

When I install using an editable install using egg, the import fails

pip install -e git+https://github.com/googleapis/python-bigquery.git@1f4ebb1eca4f9380a31172fc8cb2fae125f8c5a2#egg=google_cloud_bigquery --ignore-installed

When I install without an editable install, the import succeeds

pip install https://github.com/googleapis/python-bigquery/archive/main.zip --ignore-installed

I can produce the same issue with the google-auth package. The import of google.auth fails when I use

pip install -e git+https://github.com/googleapis/google-auth-library-python.git@7ab0fced0008ad9283c955b7dd7e2b7db55ae5e7#egg=google_auth --ignore-installed`

but not with

pip install https://github.com/googleapis/google-auth-library-python/archive/main.zip

The change to move to native namespace packages was released in July for google-auth.

We should investigate whether we want to block on this issue before reverting the change as we do want to support python 3.12 without using pkg_resources.

We could also consider the change to move to native namespace packages as a potential breaking change and bump the major version.

If this only affects editable installs, IMO we should consider this a minor version instead of a major version bump as there are limitations for editable installations:
https://setuptools.pypa.io/en/latest/userguide/development_mode.html#limitations

@parthea parthea mentioned this pull request Nov 29, 2023
Closed
@tswast
Copy link
Contributor
tswast commented Nov 29, 2023

Thanks for the investigation! I think we can remove the "-e" in the BigFrames tests.

@kiraksi kiraksi restored the remove-pkgresources branch November 29, 2023 01:53
@kiraksi kiraksi 68FA deleted the remove-pkgresources branch November 29, 2023 05:55
This was referenced Dec 5, 2023
Closed
Closed
Closed
This was referenced Dec 19, 2023
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@parthea parthea parthea approved these changes

@chalmerlowe chalmerlowe chalmerlowe approved these changes

Assignees
No one assigned
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. size: m Pull request size is medium.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Remove pkg_resource dependency after native namespace compatibility is implemented use native support for namespace packages
5 participants
@kiraksi @tswast @parthea @chalmerlowe @yoshi-kokoro
0