8000 Jorwoods/query endpoint paging by jorwoods · Pull Request #881 · tableau/server-client-python · GitHub
[go: up one dir, main page]
Skip to content

Jorwoods/query endpoint paging #881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 29 commits into from
Mar 15, 2022
Merged

Jorwoods/query endpoint paging #881

merged 29 commits into from
Mar 15, 2022

Conversation

jorwoods
Copy link
Contributor
@jorwoods jorwoods commented Sep 4, 2021

Tweaks the QuerySet added by @scuml to enable automatic pagination through all results. @scuml - Was the original intent to have users explicitly paginate through results? Would handling the pagination implicitly within the __iter__ break anything?

Also adds the ability to use QuerySet operations on Groups and Flows.

@jorwoods jorwoods force-pushed the jorwoods/query_endpoint_paging branch 2 times, most recently from 0b1f281 to af99866 Compare September 4, 2021 21:39
@scuml
Copy link
Contributor
scuml commented Sep 7, 2021

Was the original intent to have users explicitly paginate through results?

Yes - this looks good. Thanks for the improvement.

@jorwoods jorwoods force-pushed the jorwoods/query_endpoint_paging branch 3 times, most recently from 28e76a8 to 2c54a16 Compare September 18, 2021 15:36
@jorwoods
Copy link
Contributor Author

@bcantoni Seems the slack notifier is failing when I push on this PR. The error message is empty so I am not sure what is needed to resolve it.

@jorwoods jorwoods marked this pull request as draft September 18, 2021 21:41
@jorwoods jorwoods marked this pull request as ready for review September 19, 2021 15:04
@jorwoods jorwoods force-pushed the jorwoods/query_endpoint_paging branch from afd30e4 to dabe2cf Compare November 1, 2021 23:56
@salesforce-cla
Copy link
salesforce-cla bot commented Nov 1, 2021

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Jordan Woods <J***@m***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce.com Contributor License Agreement and this Pull Request will be revalidated.

@jorwoods jorwoods force-pushed the jorwoods/query_endpoint_paging branch 3 times, most recently from 2b40824 to a19588d Compare January 30, 2022 19:26
@jorwoods
Copy link
Contributor Author
jorwoods commented Feb 4, 2022

This to me is a pretty important PR that should definitely be merged before the next release. Any use of the QuerySet methods without this fix will only yield the first 100 items, regardless of how many exist.

jorwoods and others added 11 commits February 18, 2022 07:21
@jorwoods
Minimum page number is 1
7bb2834
@jorwoods
Fix slicing with negative indices
bf87aa4
@jorwoods
DRY range definitions
49d1ca2
@jorwoods
Properly support negative steps
b86ec4b
@jorwoods
Squashed commit of the following:
91fe317 
commit ad57851
Author: jorwoods <jorwoods@users.noreply.github.com>
Date:   Thu Nov 11 15:55:01 2021 -0600

    Jorwoods/type hint revisions (tableau#956)

    Add type hints and tests for workbook and data source revisions

commit bb0feb8
Author: elsherif <selsherif@tableau.com>
Date:   Wed Nov 10 01:21:16 2021 +0100

    Add support to datasource and workbook revisions (tableau#931)

    * add support to datasource and workbook revisions (get all item revisions / downlad item by revision number)

    * add type hints

    * remove wrong type hints

commit 31d420f
Author: Stephen Mitchell <scum@mac.com>
Date:   Tue Nov 9 01:02:14 2021 -0500

    Clean up hidden_views by making it an attribute of WorkbookItem (tableau#617)

    * Clean up hidden views and make it an attribute of workbookitem

    * Pycodestyle error fix for one letter variable

    Co-authored-by: Jac <jacalata@users.noreply.github.com>
    Co-authored by: Stephen Mitchell https://github.com/scuml

commit 7443f68
Author: jorwoods <jorwoods@users.noreply.github.com>
Date:   Sun Nov 7 05:57:42 2021 -0600

    Add type hints to `Job` (tableau#939)

commit c8170ae
Author: Tyler Doyle <doyle.tyler@gmail.com>
Date:   Mon Nov 1 20:36:17 2021 -0400

    Fix slack once and for all (tableau#946)

    The red X keeps coming back so I'd like to mark this as allowably fail-able -- that way the innards of Slack/OAuth/Tableau credentials don't keep polluting test run reports :)
@jorwoods
Raise error if filter receives positional arguments
0e28333
@jorwoods
Add RuntimeError for queryset positional arguments
50feca2
@jorwoods
Fix broken test
73f3f0c
@jacalata @jorwoods
make capability elements non-optional
a01e9fd
@jacalata @jorwoods
plus run black formatter
b1cf0d1
@jorwoods
Black formatting
a5b1542
@jorwoods jorwoods force-pushed the jorwoods/query_endpoint_paging branch from a19588d to a5b1542 Compare February 18, 2022 13:49
@jacalata
Copy link
Contributor

I think I'm missing something here - what does this add to the existing Pager/Pagination functionality? In the sample list.py, I can retrieve over 100 views already.

@jorwoods
Copy link
Contributor Author
jorwoods commented Mar 13, 2022 •
edited
Loading

Mostly it’s syntactic sugar. This corrects an issue with the QuerySet introduced in #615 .

It makes filtering easier than building aRequestOptions object to pass in. Searching for a given user, for example, becomes:

server.users.filter(name=‘jorwoods’)

@jorwoods
Copy link
Contributor Author

@jacalata I should clarify a little more. The QuerysetEndpoint that was introduced by #615 would only handle the first page. This PR fixes the QuerysetEndpoint so that it can work on more than the first page.

jacalata
jacalata reviewed Mar 15, 2022
View reviewed changes
@jacalata jacalata merged commit e1f861b into tableau:development Mar 15, 2022
@jorwoods jorwoods deleted the jorwoods/query_endpoint_paging branch March 16, 2022 13:20
jacalata pushed a commit that referenced this pull request Mar 30, 2022
@jorwoods @jacalata
Jorwoods/query endpoint paging (#881)
6a56f08 
* Automatically paginate in QuerySet endpoints
* Enable queryset operations for flows, groups, jobs, projects
* Keep only page_size in memory
* NULL result cache to ensure restarting from the first page
* Use modulus to simplify idx calculation
* Add support for slicing
* Minimum page number is 1
* Properly support negative steps
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jacalata jacalata jacalata left review comments

Assignees
No one assigned
Labels
cla:signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jorwoods @scuml @jacalata
0