8000 Jac/redact embedded creds by jacalata · Pull Request #1035 · tableau/server-client-python · GitHub
[go: up one dir, main page]

Skip to content

Jac/redact embedded creds #1035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.6', '3.7', '3.8', '3.9', '3.10']
python-version: ['3.7', '3.8', '3.9', '3.10']

runs-on: ${{ matrix.os }}

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Use the Tableau Server Client (TSC) library to increase your productivity as you
* Create users and groups.
* Query projects, sites, and more.

This repository contains Python source code for the library and sample files showing how to use it. Python versions 3.6 and up are supported.
This repository contains Python source code for the library and sample files showing how to use it. As of May 2022, Python versions 3.7 and up are supported.

To see sample code that works directly with the REST API (in Java, Python, or Postman), visit the [REST API Samples](https://github.com/tableau/rest-api-samples) repo.

Expand Down
Binary file removed out.pdf
Binary file not shown.
2 changes: 1 addition & 1 deletion samples/add_default_permission.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
####
# This script demonstrates how to add default permissions using TSC
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
#
# In order to demonstrate adding a new default permission, this sample will create
# a new project and add a new capability to the new project, for the default "All users" group.
Expand Down
2 changes: 1 addition & 1 deletion samples/create_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to create a group using the Tableau
# Server Client.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####


Expand Down
2 changes: 1 addition & 1 deletion samples/create_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# parent_id.
#
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/create_schedules.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to create schedules using the Tableau
# Server Client.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####


Expand Down
2 changes: 1 addition & 1 deletion samples/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to export a view using the Tableau
# Server Client.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/filter_sort_groups.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to filter and sort groups using the Tableau
# Server Client.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####


Expand Down
2 changes: 1 addition & 1 deletion samples/filter_sort_projects.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to use the Tableau Server Client
# to filter and sort on the name of the projects present on site.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/kill_all_jobs.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to kill all of the running jobs
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/list.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to list all of the workbooks or datasources
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/login.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to log in to Tableau Server Client.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/metadata_query.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to use the metadata API to query information on a published data source
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/move_workbook_projects.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# a workbook that matches a given name and update it to be in
# the desired project.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/move_workbook_sites.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# a workbook that matches a given name, download the workbook,
# and then publish it to the destination site.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/publish_datasource.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# more information on personal access tokens, refer to the documentations:
# (https://help.tableau.com/current/server/en-us/security_personal_access_tokens.htm)
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/publish_workbook.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# For more information, refer to the documentations on 'Publish Workbook'
# (https://onlinehelp.tableau.com/current/api/rest_api/en-us/help.htm)
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/query_permissions.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
####
# This script demonstrates how to query for permissions using TSC
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
#
# Example usage: 'python query_permissions.py -s https://10ax.online.tableau.com --site
# devSite123 -u tabby@tableau.com workbook b4065286-80f0-11ea-af1b-cb7191f48e45'
Expand Down
2 changes: 1 addition & 1 deletion samples/refresh.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to use trigger a refresh on a datasource or workbook
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/refresh_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to use the Tableau Server Client
# to query extract refresh tasks and run them as needed.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
2 changes: 1 addition & 1 deletion samples/set_refresh_schedule.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This script demonstrates how to set the refresh schedule for
# a workbook or datasource.
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####


Expand Down
2 changes: 1 addition & 1 deletion samples/update_connection.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
####
# This script demonstrates how to update a connections credentials on a server to embed the credentials
#
# To run the script, you must have installed Python 3.6 or later.
# To run the script, you must have installed Python 3.7 or later.
####

import argparse
Expand Down
8 changes: 6 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# This makes work easier for offline installs or low bandwidth machines
needs_pytest = {'pytest', 'test', 'ptr'}.intersection(sys.argv)
pytest_runner = ['pytest-runner'] if needs_pytest else []
test_requirements = ['black', 'mock', 'pytest', 'requests-mock>=1.0,<2.0', 'mypy==0.910']
test_requirements = ['black', 'mock', 'pytest', 'requests-mock>=1.0,<2.0', 'mypy>=0.920']

setup(
name='tableauserverclient',
Expand All @@ -25,7 +25,10 @@
author_email='github@tableau.com',
url='https://github.com/tableau/server-client-python',
package_data={'tableauserverclient':['py.typed']},
packages=['tableauserverclient', 'tableauserverclient.models', 'tableauserverclient.server',
packages=['tableauserverclient',
'tableauserverclient.helpers',
'tableauserverclient.models',
'tableauserverclient.server',
'tableauserverclient.server.endpoint'],
license='MIT',
description='A Python module for working with the Tableau Server REST API.',
Expand All @@ -37,6 +40,7 @@
'defusedxml>=0.7.1',
'requests>=2.11,<3.0',
],
python_requires='>3.7.0',
tests_require=test_requirements,
extras_require={
'test': test_requirements
Expand Down
1 change: 1 addition & 0 deletions tableauserverclient/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
NotSignedInError,
Pager,
)
from .helpers import *

__version__ = get_versions()["version"]
__VERSION__ = __version__
Expand Down
1 change: 1 addition & 0 deletions tableauserverclient/helpers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .strings import *
65 changes: 65 additions & 0 deletions tableauserverclient/helpers/strings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
import requests

from defusedxml.ElementTree import fromstring, tostring
from functools import singledispatch
from typing import TypeVar


# the redact method can handle either strings or bytes, but it can't mix them.
# Generic type so we can write the actual logic once, then use singledispatch to
# create the replacement text with the correct type
T = TypeVar("T", str, bytes)


# TODO: ideally this would be in the logging config
def safe_to_log(server_response: requests.Response) -> str:
"""Checks if the server_response content is not xml (eg binary image or zip)
and replaces it with a constant"""
ALLOWED_CONTENT_TYPES = ("application/xml", "application/xml;charset=utf-8")
if server_response.headers.get("Content-Type", None) not in ALLOWED_CONTENT_TYPES:
return "[Truncated File Contents]"

""" Check to determine if the response is a text response (xml or otherwise)
so that we do not attempt to log bytes and other binary data. """
if not server_response.content or not server_response.encoding:
return ""
# max length 1000
loggable_response: str = server_response.content.decode(server_response.encoding)[:1000]
redacted_response: str = redact_xml(loggable_response)
return redacted_response


# usage: _redact_any_type("<xml workbook password= cooliothesecond>")
# -> b"<xml workbook password =***************">
def _redact_any_type(xml: T, sensitive_word: T, replacement: T, encoding=None) -> T:
try:
root = fromstring(xml)
matches = root.findall(".//*[@password]")
for item in matches:
item.attrib["password"] = "********"
matches = root.findall(".//password")
for item in matches:
item.text = "********"
# tostring returns bytes unless an encoding value is passed
return tostring(root, encoding=encoding)
except Exception:
# something about the xml handling failed. Just cut off the text at the first occurrence of "password"
location = xml.find(sensitive_word)
return xml[:location] + replacement


@singledispatch
def redact_xml(content):
# this will only be called if it didn't get directed to the str or bytes overloads
raise TypeError("Redaction only works on xml saved as str or bytes")


@redact_xml.register
def _(xml: str) -> str:
out = _redact_any_type(xml, "password", "...[redacted]", encoding="unicode")
return out


@redact_xml.register # type: ignore[no-redef]
def _(xml: bytes) -> bytes:
return _redact_any_type(bytearray(xml), b"password", b"..[redacted]")
1 change: 1 addition & 0 deletions tableauserverclient/server/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,4 @@
from .server import Server
from .pager import Pager
from .exceptions import NotSignedInError
from ..helpers import *
Loading
0