8000 Error publishing large twbx files · Issue #754 · tableau/server-client-python · GitHub
[go: up one dir, main page]
Skip to content

Error publishing large twbx files #754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
AnandShiva opened this issue Dec 7, 2020 · 8 comments
Closed

Error publishing large twbx files #754

AnandShiva opened this issue Dec 7, 2020 · 8 comments
Labels
bug

Comments

@AnandShiva
Copy link

While publishing a large twbx file with latest 0.14 version of tableau-server-client using the below code,

wb_item = server.workbooks.publish(wb_item,target_file,'Overwrite')

I am getting the following error.

File "/usr/local/lib/python3.9/site-packages/tableauserverclient/server/endpoint/fileuploads_endpoint.py", line 45, in read_chunks

chunked_content = file.read(CHUNK_SIZE)

ValueError: read of closed file

The same code works for 0.13 and below versions. And also it works for smaller workbooks, even with extracts.
@bcantoni
Copy link
Contributor
bcantoni commented Dec 7, 2020

@AnandShiva thanks for reporting this. Can you confirm what version of Tableau Server you are connecting to?

Also, do you have a sense of what size is causing you the problem? If nothing else, maybe you can give the size of the one which fails and the size of a smaller one which passes.

@AnandShiva
Copy link
Author
AnandShiva commented Dec 7, 2020 •
edited
Loading

@bcantoni I tried with the file of size 800MB and it failed. When I tried with a workbook of 70 MB it passed.
I am facing this issue with the latest 0.14 version of the package. The tableau server is the 2020.03 version.

@bcantoni
Copy link
Contributor
bcantoni commented Dec 7, 2020

Ok thanks, that's helpful.

This might be related to changes which came in to 0.14 via #704.

@avishaymahluf
Copy link

Hi,
I'm hitting the same issue when trying to publish a large hyper file (1.2GB).
client version - 0.14, server version - 2020.3.1.

@noobling
Copy link
noobling commented Dec 8, 2020

Do we have a resolution for this? It renders the package pretty unusable...

@bcantoni bcantoni added the bug label Dec 8, 2020
@bcantoni
Copy link
Contributor
bcantoni commented Dec 8, 2020

I can repro this as well. It seems like any file >64MB will trigger this, so this path likely wasn't tested as part of #704.

A possible workaround until this is fixed is to stick with v0.13 of the library.

@AnandShiva
Copy link
Author

One thing which I am doing to still keep using the 0.14 package and still not face the issue is, create a small size twbx extract workbook file (~70MB) pointing to a temp smaller data source, change the data source to actual data source programmatically via tableau document API, and publish that.
After publishing trigger workbook refresh via server.workbooks.refresh(workbook) which will update the workbook to larger data source extract clearing older smaller data source.

Or of course, you can just do pip3 install tableauserverclient==0.13 and continue what you are doing anyway.

shinchris pushed a commit that referenced this issue Dec 9, 2020
Fixes issue #754 by moving file read logic inside generator
e624178
@shinchris shinchris mentioned this issue Dec 9, 2020
Merged
shinchris pushed a commit that referenced this issue Dec 9, 2020
Merge pull request #757 from tableau/fix_754
a81562d 
Fixes issue #754 by moving file read logic inside generator
shinchris pushed a commit that referenced this issue Dec 10, 2020
Merge pull request #757 from tableau/fix_754
41fd8f2 
Fixes issue #754 by moving file read logic inside generator
@shinchris
Copy link
Contributor

Fix for this is released with v0.14.1.

shinchris pushed a commit that referenced this issue Feb 5, 2021
@tjones-commits @maddy-at-leisure
Update site properties and functions (#777)
857199b 
* Update publish_workbook.py (#694)

* Update publish_workbook.py

Added below arguments, without this there is a sign-in error on publishing a test file to Tableau Online

parser.add_argument('--sitename', '-S', default='', help='sitename required')
tableau_auth = TSC.TableauAuth(args.username, password,site_id=args.sitename)

* Update publish_workbook.py

Edits (as requested) to publish workbooks on Tableau Online which removes the Sign-in Error.

* Update publish_workbook.py

* Merge pull request #745 from tableau/fix_732

Server versions before 2020.1 do not accept encoded query param delimiters

* Merge pull request #757 from tableau/fix_754

Fixes issue #754 by moving file read logic inside generator

* Updates changelog for v0.14.1

* update the site item to reflect api response

* update test model

* remove extra test assets

* trimming line length

* unit test all properties. fix some properties. Remove extra code

* make requested changes

* make requested changes

Co-authored-by: Chris Shin <cshin@tableau.com>
Co-authored-by: Madhura Selvarajan <MADHURA0910@GMAIL.COM>
Co-authored-by: Terrence Jones <tjones@arkatechture.com>
shinchris pushed a commit that referenced this issue Feb 16, 2021
@jayvdb @maddy-at-leisure
MANIFEST.in: Add docs and test data (#780)
f64fcf9 
* Update publish_workbook.py (#694)

* Update publish_workbook.py

Added below arguments, without this there is a sign-in error on publishing a test file to Tableau Online

parser.add_argument('--sitename', '-S', default='', help='sitename required')
tableau_auth = TSC.TableauAuth(args.username, password,site_id=args.sitename)

* Update publish_workbook.py

Edits (as requested) to publish workbooks on Tableau Online which removes the Sign-in Error.

* Update publish_workbook.py

* Merge pull request #745 from tableau/fix_732

Server versions before 2020.1 do not accept encoded query param delimiters

* Merge pull request #757 from tableau/fix_754

Fixes issue #754 by moving file read logic inside generator

* Updates changelog for v0.14.1

* MANIFEST.in: Add docs and test data

Closes #779

Co-authored-by: Chris Shin <cshin@tableau.com>
Co-authored-by: Madhura Selvarajan <MADHURA0910@GMAIL.COM>
jorwoods added a commit to jorwoods/server-client-python that referenced this issue Feb 18, 2021
@jorwoods
Squashed commit of the following:
7efb7a6 
commit 2609e27
Merge: d107c48 3a8ec6c
Author: Chris Shin <cshin@tableau.com>
Date:   Wed Feb 17 16:00:42 2021 -0800

    Merge pull request tableau#803 from tableau/release-0.15

    syncing development branch with v0.15 changelog

commit d107c48
Author: Tyler Doyle <doyle.tyler@gmail.com>
Date:   Wed Feb 17 15:38:14 2021 -0800

    Add Mypy to CI runs (tableau#802)

    Add mypy runs to CI, but skip misc errors, so we only see the important stuff. For now these are non blocking.

commit 3a8ec6c
Merge: 004ab31 a5b9252
Author: Chris Shin <cshin@tableau.com>
Date:   Tue Feb 16 16:51:37 2021 -0800

    Merge branch 'master' into release-0.15

commit 004ab31
Author: Chris Shin <cshin@tableau.com>
Date:   Tue Feb 16 16:49:19 2021 -0800

    Updates changelog and contributors list for v0.15

commit 6c7a87b
Author: Chris Shin <cshin@tableau.com>
Date:   Tue Feb 16 15:09:45 2021 -0800

    Removes travis and adds linting/testing into github action (tableau#798)

    * Removes travis and adds github workflow

    * Addressing code review feedback

commit 9179637
Author: Lee Boynton <lee@lboynton.com>
Date:   Tue Feb 16 23:09:22 2021 +0000

    Add support for getting groups that a user belongs to (tableau#799)

    * Add support for getting groups that a user belongs to

    * Use more descriptive name for pager function

commit fe992ee
Author: Chris Shin <cshin@tableau.com>
Date:   Tue Feb 16 14:33:59 2021 -0800

    [Tasks] Translate task type from server to TSC enum (tableau#796)

    * Adds task type mapping to translate server response

    * Updates tests to match server response for task type

    * Fixes pycodestyle error

commit f64fcf9
Author: John Vandenberg <jayvdb@gmail.com>
Date:   Wed Feb 17 01:21:46 2021 +0800

    MANIFEST.in: Add docs and test data (tableau#780)

    * Update publish_workbook.py (tableau#694)

    * Update publish_workbook.py

    Added below arguments, without this there is a sign-in error on publishing a test file to Tableau Online

    parser.add_argument('--sitename', '-S', default='', help='sitename required')
    tableau_auth = TSC.TableauAuth(args.username, password,site_id=args.sitename)

    * Update publish_workbook.py

    Edits (as requested) to publish workbooks on Tableau Online which removes the Sign-in Error.

    * Update publish_workbook.py

    * Merge pull request tableau#745 from tableau/fix_732

    Server versions before 2020.1 do not accept encoded query param delimiters

    * Merge pull request tableau#757 from tableau/fix_754

    Fixes issue tableau#754 by moving file read logic inside generator

    * Updates changelog for v0.14.1

    * MANIFEST.in: Add docs and test data

    Closes tableau#779

    Co-authored-by: Chris Shin <cshin@tableau.com>
    Co-authored-by: Madhura Selvarajan <MADHURA0910@GMAIL.COM>

commit 88a0188
Author: Chris Shin <cshin@tableau.com>
Date:   Tue Feb 16 09:17:35 2021 -0800

    [Subscriptions] Add new fields and ability to update (tableau#794)

    * Add fields and parsing logic

    * Update subscription create request

    * Adds update request to subscriptions

    * Changes subscription change request creation to use tsrequest annotation

    * Update tests for parsing new fields

    * Fixes codestyle issues

    * Removes user and schedule name

    * Fixes test failure

commit 026bca8
Author: Chris Shin <cshin@tableau.com>
Date:   Thu Feb 11 15:34:08 2021 -0800

    Adds skipConnectionCheck to publish workbook (tableau#791)

    * Adds skipConnectionCheck flag to publish workbook

    * Removes unnecessary lines

    * Fixes style error

commit 857199b
Author: tjones-commits <70481977+tjones-commits@users.noreply.github.com>
Date:   Fri Feb 5 17:09:32 2021 -0500

    Update site properties and functions (tableau#777)

    * Update publish_workbook.py (tableau#694)

    * Update publish_workbook.py

    Added below arguments, without this there is a sign-in error on publishing a test file to Tableau Online

    parser.add_argument('--sitename', '-S', default='', help='sitename required')
    tableau_auth = TSC.TableauAuth(args.username, password,site_id=args.sitename)

    * Update publish_workbook.py

    Edits (as requested) to publish workbooks on Tableau Online which removes the Sign-in Error.

    * Update publish_workbook.py

    * Merge pull request tableau#745 from tableau/fix_732

    Server versions before 2020.1 do not accept encoded query param delimiters

    * Merge pull request tableau#757 from tableau/fix_754

    Fixes issue tableau#754 by moving file read logic inside generator

    * Updates changelog for v0.14.1

    * update the site item to reflect api response

    * update test model

    * remove extra test assets

    * trimming line length

    * unit test all properties. fix some properties. Remove extra code

    * make requested changes

    * make requested changes

    Co-authored-by: Chris Shin <cshin@tableau.com>
    Co-authored-by: Madhura Selvarajan <MADHURA0910@GMAIL.COM>
    Co-authored-by: Terrence Jones <tjones@arkatechture.com>

commit a5b9252
Merge: bcb881c 1fc349c
Author: Chris Shin <cshin@tableau.com>
Date:   Thu Dec 10 09:29:42 2020 -0800

    Merge pull request tableau#759 from tableau/0.14.0-patch

    v0.14.1 patch release
    * Fixed filter query issue for server version below 2020.1 (tableau#745)
    * Fixed large workbook/datasource publish issue (tableau#757)

commit 1fc349c
Author: Chris Shin <cshin@tableau.com>
Date:   Wed Dec 9 16:30:03 2020 -0800

    Updates changelog for v0.14.1

commit 41fd8f2
Author: Chris Shin <cshin@tableau.com>
Date:   Wed Dec 9 15:49:48 2020 -0800

    Merge pull request tableau#757 from tableau/fix_754

    Fixes issue tableau#754 by moving file read logic inside generator

commit 138476f
Author: Chris Shin <cshin@tableau.com>
Date:   Wed Dec 9 15:49:23 2020 -0800

    Merge pull request tableau#745 from tableau/fix_732

    Server versions before 2020.1 do not accept encoded query param delimiters

commit bcb881c
Merge: 8d51355 1e089b4
Author: Chris Shin <cshin@tableau.com>
Date:   Mon Nov 9 09:19:23 2020 -0800

    Merge pull request tableau#725 from tableau/development

    Syncing master with v0.14.0 changes from development.

commit 8d51355
Author: Madhura Selvarajan <MADHURA0910@GMAIL.COM>
Date:   Wed Sep 23 10:45:27 2020 -0400

    Update publish_workbook.py (tableau#694)

    * Update publish_workbook.py

    Added below arguments, without this there is a sign-in error on publishing a test file to Tableau Online

    parser.add_argument('--sitename', '-S', default='', help='sitename required')
    tableau_auth = TSC.TableauAuth(args.username, password,site_id=args.sitename)

    * Update publish_workbook.py

    Edits (as requested) to publish workbooks on Tableau Online which removes the Sign-in Error.

    * Update publish_workbook.py

commit 273af7a
Merge: 43e1b06 168489c
Author: Chris Shin <cshin@tableau.com>
Date:   Wed Sep 2 14:39:13 2020 -0700

    Merge pull request tableau#685 from tableau/development

    [v0.13] Merge development into master

    Added notes field to JobItem (tableau#571)
    Added webpage_url field to WorkbookItem (tableau#661)
    Added support for switching between sites (tableau#655)
    Added support for querying favorites for a user (tableau#656)
    Added support for Python 3.8 (tableau#659)
    Added support for Data Alerts (tableau#667)
    Added support for basic Extract operations - Create, Delete, en/re/decrypt for site (tableau#672)
    Added support for creating and querying Active Directory groups (tableau#674)
    Added support for asynchronously updating a group (tableau#674)
    Improved handling of invalid dates (tableau#529)
    Improved consistency of update_permission endpoints (tableau#668)
    Documentation updates (tableau#658, tableau#669, tableau#670, tableau#673, tableau#683)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@bcantoni @shinchris @AnandShiva @avishaymahluf @noobling
0