8000 feat: Extend GraphQL class to support multiple authentication methods… by timmy61109 · Pull Request #3177 · python-gitlab/python-gitlab · GitHub
[go: up one dir, main page]

Skip to content

feat: Extend GraphQL class to support multiple authentication methods… #3177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

timmy61109
Copy link
@timmy61109 timmy61109 commented Apr 18, 2025

… and more API options

This commit extends the constructor (__init__ method) of the GraphQL class to handle a wider range of interactions with the GitLab API. The main changes include:

  • Added support for multiple authentication tokens: In addition to the original token, it now supports private_token, oauth_token, and job_token, allowing users to configure based on different authentication requirements.
  • Added HTTP Basic Authentication: Introduced http_username and http_password parameters, allowing the use of HTTP Basic Authentication to interact with the API.
  • Added API version control: Introduced the api_version parameter, with a default value of "4", allowing users to specify the GitLab API version to use.
  • Added pagination control parameters: Added per_page, pagination, and order_by parameters to provide finer control over the API's pagination behavior and data sorting.
  • Added an option to keep the Base URL: Added the keep_base_url parameter, allowing the configured base URL to be retained across multiple API calls.
  • Added **kwargs: Any: Allows passing additional keyword arguments, providing greater flexibility.

These changes make the GraphQL class more powerful and flexible, capable of handling a broader range of GitLab API use cases.

Changes

Documentation and testing

Please consider whether this PR needs documentation and tests. This is not required, but highly appreciated:

@timmy61109 timmy61109 force-pushed the main branch 2 times, most recently from b9041dd to b199086 Compare April 23, 2025 20:42
### Features

- **group**: Add can use config file.
([`bf7eba5`](https://github.com/python-gitlab/python-gitlab/pull/3177/commits/bf7eba5909669e19757ef1c5b590f613be00d5f4))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semantic release will automatically create changelog entries. So should not modify this file.

10000
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I made a change because Python 3 detection requires a new version number. I'll either remove this change before the official merge or rebase to your latest version and then use your version number.

@@ -3,4 +3,4 @@
__email__ = "gauvainpocentek@gmail.com"
__license__ = "LGPL3"
__title__ = "python-gitlab"
__version__ = "5.6.0"
__version__ = "5.7.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semantic release will automatically update the version number. So should not modify this file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I made a change because Python 3 detection requires a new version number. I'll either remove this change before the official merge or rebase to your latest version and then use your version number.

…n methods and more API options

This commit extends the constructor (`__init__` method) of the `GraphQL` class to handle a wider range of interactions with the GitLab API. The main changes include:

- **Added support for multiple authentication tokens:** In addition to the original `token`, it now supports `private_token`, `oauth_token`, and `job_token`, allowing users to configure based on different authentication requirements.
- **Added HTTP Basic Authentication:** Introduced `http_username` and `http_password` parameters, allowing the use of HTTP Basic Authentication to interact with the API.
- **Added API version control:** Introduced the `api_version` parameter, with a default value of `"4"`, allowing users to specify the GitLab API version to use.
- **Added pagination control parameters:** Added `per_page`, `pagination`, and `order_by` parameters to provide finer control over the API's pagination behavior and data sorting.
- **Added an option to keep the Base URL:** Added the `keep_base_url` parameter, allowing the configured base URL to be retained across multiple API calls.
- **Added `**kwargs: Any`:** Allows passing additional keyword arguments, providing greater flexibility.

These changes make the `GraphQL` class more powerful and flexible, capable of handling a broader range of GitLab API use cases.
…ration files

This commit introduces a new class method from_config to the GraphQL class.

This method allows users to create a Gitlab connection object by reading configuration files. Users can specify the ID and paths of the configuration files, and the method will parse the connection information (e.g., URL, tokens, SSL verification, etc.) from the files to create a pre-configured Gitlab object.

This feature provides a more convenient and configurable way to manage GitLab connections, especially useful for scenarios requiring connections to multiple GitLab instances or aiming to separate connection configurations from the codebase.

Example usage (assuming the configuration file is gitlab.ini):

```python
gl = GraphQL.from_config(config_files=['gitlab.ini'])
```

Internally, this method utilizes gitlab.config.GitlabConfigParser to handle the parsing of the configuration files.
…iguration files

This commit introduces a new class method from_config to the GraphQL class.

This method allows users to create a Gitlab connection object by reading configuration files. Users can specify the ID and paths of the configuration files, and the method will parse the connection information (e.g., URL, tokens, SSL verification, etc.) from the files to create a pre-configured Gitlab object.

This feature provides a more convenient and configurable way to manage GitLab connections, especially useful for scenarios requiring connections to multiple GitLab instances or aiming to separate connection configurations from the codebase.

Example usage (assuming the configuration file is gitlab.ini):
Python

gl = GraphQL.from_config(config_files=['gitlab.ini'])

Internally, this method utilizes gitlab.config.GitlabConfigParser to handle the parsing of the configuration files.
Upgraded version to allow successful installation of packages into the library, as the previous version was not working.
Added parameters to be compatible with parameters required by from_config.
Added parameters to be compatible with parameters required by from_config.
Added parameters to be compatible with parameters required by from_config.
Adding this feature would directly allow specific conditions to be variables, increasing the flexibility of application.

Using variables: https://gql.readthedocs.io/en/stable/usage/variables.html
@timmy61109
Copy link
Author

@JohnVillalovos Regarding Current Mypy Type Checking Issues in python-gitlab Project

I'm currently facing two issues in the project that require some assistance, both related to mypy type checking. These issues might impact code robustness and future extensibility:

  1. Incompatible Types in Assignment (gitlab/client.py:1426)
    When running tox for type checking, mypy reports an Incompatible types in assignment error on line 1426 of gitlab/client.py. Specifically, during the initialization of GraphQL(_BaseGraphQL).init and AsyncGraphQL.init, mypy indicates that the type of the assigned expression (gql.client.Client) is incompatible with the variable's expected type (httpx._client.Client | None).
    This discrepancy seems to stem from the type assignment self.client = client within GitlabTransport.init in python-gitlab/gitlab/_backends/graphql.py. While this type mismatch doesn't seem to affect runtime behavior, I believe it's a type hinting issue that likely requires adjustments to the relevant test code or type declarations, rather than a fundamental logic error.
  2. Missing GraphQL Query Argument and Test Writing Challenges
    To enhance our GraphQL query capabilities, I've added a variable_values parameter to the execute method within the GraphQL class. The aim is to support passing GraphQL variables directly into queries, enabling resource retrieval with variable-based requests. However, mypy is flagging a Missing positional argument "variable_values" error on line 19 of tests/functional/api/test_graphql.py. This indicates that our existing test code is not correctly providing this newly added argument when calling the execute method.
    I'm not yet proficient in writing text (unit or integration) tests to validate the functionality of this variable_values parameter, nor am I sure how to ensure mypy passes its checks successfully for this change. I would appreciate your guidance or assistance in this area.

Although I've currently installed my modified package and it's working fine, I'm keen to contribute to this project. Therefore, I'd like to resolve the issues I mentioned above.

I look forward to your support and advice on these matters. Thank you.

class GraphQL(_BaseGraphQL):
    def __init__(
        self,
        url: str | None = None,
        *,
        private_token: str | None = None,
        oauth_token: str | None = None,
        job_token: str | None = None,
        ssl_verify: bool | str = True,
        client: httpx.Client | None = None,
        http_username: str | None = None,
        self._http_client = client or httpx.AsyncClient(**self._client_opts)
        self._transport = GitlabAsyncTransport(self._url, client=self._http_client)
        self._client = gql.Client(
            transport=self._transport,
            fetch_schema_from_transport=fetch_schema_from_transport,
        )
        self._gql = gql.gql
       #: Create a session object for requests
        _backend: type[_backends.DefaultBackend] = kwargs.pop(
            "backend", _backends.DefaultBackend
        )
        self._backend = _backend(**kwargs)
        self.session = self._backend.client

        self.per_page = per_page
        self.pagination = pagination
        self.order_by = order_by

Error log:

tests/install/test_install.py:6: error: Unused "type: ignore" comment  [unused-ignore]
tests/functional/cli/test_cli_variables.py:4: error: Cannot find implementation or library stub for module named "responses"  [import-not-found]
tests/functional/cli/test_cli_projects.py:5: error: Cannot find implementation or library stub for module named "responses"  [import-not-found]
tests/functional/cli/conftest.py:2: error: Cannot find implementation or library stub for module named "responses"  [import-not-found]
gitlab/client.py:1426: error: Incompatible types in assignment (expression has type "gql.client.Client", variable has type "httpx._client.Client | None")  [assignment]
gitlab/client.py:1454: error: Item "Client" of "Client | None" has no attribute "execute"  [union-attr]
gitlab/client.py:1454: error: Item "None" of "Client | None" has no attribute "execute"  [union-attr]
gitlab/client.py:1499: error: Incompatible return value type (got "GraphQL", expected "Gitlab")  [return-value]
gitlab/client.py:1670: error: Incompatible types in assignment (expression has type "gql.client.Client", variable has type "httpx._client.Client | None")  [assignment]
gitlab/client.py:1698: error: Item "Client" of "Client | None" has no attribute "execute_async"  [union-attr]
gitlab/client.py:1698: error: Item "None" of "Client | None" has no attribute "execute_async"  [union-attr]
gitlab/client.py:1743: error: Incompatible return value type (got "AsyncGraphQL", expected "Gitlab")  [return-value]
gitlab/cli.py:352: error: Unused "type: ignore" comment  [unused-ignore]
gitlab/v4/cli.py:458: error: Library stubs not installed for "yaml"  [import-untyped]
gitlab/v4/cli.py:458: note: Hint: "python3 -m pip install types-PyYAML"
gitlab/v4/cli.py:458: note: (or run "mypy --install-types" to install all missing stub packages)
gitlab/v4/cli.py:458: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
tests/functional/cli/test_cli.py:10: error: Cannot find implementation or library stub for module named "responses"  [import-not-found]
tests/functional/cli/test_cli.py:11: error: Library stubs not installed for "yaml"  [import-untyped]
tests/functional/api/test_graphql.py:19: error: Missing positional argument "variable_values" in call to "execute" of "GraphQL"  [call-arg]
tests/functional/api/test_graphql.py:27: error: Missing positional argument "variable_values" in call to "execute" of "AsyncGraphQL"  [call-arg]
tests/functional/conftest.py:151: error: Name "pytest.Config" is not defined  [name-defined]
Found 19 errors in 10 files (checked 246 source files)

@JohnVillalovos JohnVillalovos requested a review from Copilot June 10, 2025 04:22
Copilot

This comment was marked as outdated.

@JohnVillalovos JohnVillalovos requested a review from Copilot June 10, 2025 04:26
Copy link
@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR extends the GraphQL class to support multiple authentication methods, API version control, pagination parameters, and HTTP Basic Authentication, making it more versatile when interacting with the GitLab API.

  • Extended authentication options (multiple tokens and HTTP Basic credentials)
  • Added API version control and pagination parameters
  • Updated tests to exercise the new query patterns and error scenarios

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File Description
tox.ini Updated excluded paths for linting improvements
tests/unit/test_graphql.py Modified tests to use a new query and variable values for improved coverage of the new API options
gitlab/_version.py Adjusted version number, which appears inconsistent with the feature changes
CHANGELOG.md Added changelog entries reflecting the new version changes

@@ -3,4 +3,4 @@
__email__ = "gauvainpocentek@gmail.com"
__license__ = "LGPL3"
__title__ = "python-gitlab"
__version__ = "6.0.0"
__version__ = "5.7.1"
Copy link
Preview
Copilot AI Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version number was downgraded from 6.0.0 to 5.7.1, which is inconsistent with the feature additions in this PR. Please update the version number to appropriately reflect the new feature set.

Suggested change
__version__ = "5.7.1"
__version__ = "6.1.0"

Copilot uses AI. Check for mistakes.

query:

query get_projects($first: Int = 2) {
group(fullPath: "treeocean") {
Copy link
Preview
Copilot AI Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an inconsistency between the docstring (which references group 'treeocean') and the actual query (which uses 'python-gitlab'). Align these values to avoid confusion during test execution.

Suggested change
group(fullPath: "treeocean") {
group(fullPath: "python-gitlab") {

Copilot uses AI. Check for mistakes.

@nejch
Copy link
Member
nejch commented Jun 10, 2025

@timmy61109 thanks for this PR.

I'm not sure if some of or how much of this was made with the help of AI, but most of the changes do not apply to the GraphQL client. There is only a single way to authenticate against the API when using header auth, which we do (see https://docs.gitlab.com/api/graphql/#header-authentication). Likewise pagination and other things are not done the same way for GraphQL.

Could you maybe first elaborate on what you're trying to achieve and we can discuss this (possibly in an issue/discussion)?

@timmy61109
Copy link
Author
timmy61109 commented Jun 16, 2025

I'm not sure if some of or how much of this was made with the help of AI, but most of the changes do not apply to the GraphQL client. There is only a single way to authenticate against the API when using header auth, which we do (see https://docs.gitlab.com/api/graphql/#header-authentication). Likewise pagination and other things are not done the same way for GraphQL.

Could you maybe first elaborate on what you're trying to achieve and we can discuss this (possibly in an issue/discussion)?

@nejch @JohnVillalovos 裡面只有 git commit 是 AI 協助幫我翻譯完成,因為我的英文不是很好。為了可以快速可以用,因此我只是將 GitLab API Class 裡面功能參考與複製到 GraphQL API Class 裡面,並盡可能修改到可以使用。

我主要希望有以下目標:

  1. 可以使用 GraphQL ,因為 GitLab 有些新功能 RSET API 目前尚未製作好,很多都是需要從 GraphQL 取得
  2. 可以直接透過 from_config 取得資料,其實這個是我目前開這個合併請求的主要原因,我可以直接從 gl.cfg 匯入金鑰,這樣我不會遺忘要將金鑰刪除
  3. 由於現在 GraphQL 其實都是要自己寫指令,那這樣使用 Python 設計好取得資料的方式優勢就沒有,因此我希望未來可以有現成的 function 可以使用
  4. 目前我製作的 function 是 timelogs ,我目前還在自我測試中,程式可能也沒有設計的很好,可以參考: https://gitlab.atcatw.org/atca/community-edition/workflow-automation-design/-/blob/9-csv-3/workflow_automation_design/expenses.py?ref_type=heads

Only the git commit part was translated with AI assistance because my English isn't very good. To make it usable quickly, I simply referenced and copied the functionality from the GitLab API Class into the GraphQL API Class and modified it as much as possible to make it usable.

My main goals are as follows:

  1. To use GraphQL because some of GitLab's new features, such as the RSET API, are not yet available, and many require data to be retrieved from GraphQL.
  2. Be able to retrieve data directly via from_config. This is actually the main reason I opened this merge request. I can directly import the key from gl.cfg, so I won't forget to delete the key.
  3. Since GraphQL currently requires writing commands manually, the advantage of using Python to design a data retrieval method is lost. Therefore, I hope there will be pre-built functions available in the future.
  4. The function I've created so far is timelogs. I'm still testing it myself, and the code might not be well-designed yet. You can check it out here: https://gitlab.atcatw.org/atca/community-edition/workflow-automation-design/-/blob/9-csv-3/workflow_automation_design/expenses.py?ref_type=heads

Translated with DeepL.com (free version)


Only git commit was translated by AI because my English is not very good. In order to make it usable quickly, I just copied the function references in GitLab API Class to GraphQL API Class and modified it as much as possible to make it usable.

I mainly hope to achieve the following goals:

  1. Use GraphQL, because some new features of GitLab, RSET API, are not yet ready, and many of them need to be obtained from GraphQL
  2. Data can be obtained directly through from_config. In fact, this is the main reason why I opened this merge request. I can import the key directly from gl.cfg, so I won’t forget to delete the key
  3. Since GraphQL actually requires you to write your own instructions, there is no advantage in using Python to design a way to obtain data. Therefore, I hope that there will be ready-made functions available in the future
  4. The function I have created is timelogs. I am still testing it myself, and the program may not be designed very well. You can refer to: https://gitlab.atcatw.org/atca/community-edition/workflow-automation-design/-/blob/9-csv-3/workflow_automation_design/expenses.py?ref_type=heads

Translated with Google Translate

@timmy61109
Copy link
Author
timmy61109 commented Jun 16, 2025

@nejch @JohnVillalovos You've discovered a great improvement! Adding variable_values to the execute function significantly enhances its flexibility and reusability. This allows for dynamic input without altering the core logic.

gq.execute(  # type: ignore[attr-defined]
                query,
                variable_values=self.variable_values,
            )

Execution status:

query get_timeslogs(
        $group_fullpath: ID! = "atca",
        $projects_first: Int = 1,
        $projects_after: String = "eyJpZCI6IjQyMCJ9",
        $timelogs_first: Int = 100,
        $timelogs_after: String = ""
        ) {
    group(fullPath: $group_fullpath) {
        projects(
                first: $projects_first,
                after: $projects_after,
                includeSubgroups: true
                ) {
            count
            nodes {
                timelogs(first: $timelogs_first, after: $timelogs_after) {
                    totalSpentTime
                    nodes {
                        id
                        mergeRequest {

                            projectId
                            id
                            name
                            iid
                            webPath
                            webUrl
                            state

                            timeEstimate
                            totalTimeSpent
                            humanTimeEstimate
                            humanTotalTimeSpent

                            labels {
                                edges {
                                    node {
                                        title
                                    }
                                }
                            }
                            author {
                                username
                            }
                            assignees(after: "", first: 100) {
                                nodes {
                                    username
                                }
                            }

                            createdAt
                            closedAt
                            updatedAt
                            mergedAt
                            preparedAt

                            approved
                            draft
                            mergeStatusEnum
                            mergeOngoing
                            mergeableDiscussionsState

                            reviewers {
                                edges {
                                    node {
                                        username
                                    }
                                }
                            }
                            approvedBy {
                                edges {
                                    node {
                                        username
                                    }
                                }
                            }
                            mergeUser {
                                username
                            }
                        }
                        spentAt
                        summary
                        timeSpent
                        user {
                            username
                        }
                      
                    }
                    pageInfo {
                        endCursor
                      	startCursor
                        hasNextPage
                      	hasPreviousPage
                    }
                }
            }
            pageInfo {
                endCursor
                startCursor
                hasNextPage
                hasPreviousPage
            }
        }
    }
}

Execution 9A4B Result:

{
  "data": {
    "group": {
      "projects": {
        "count": null,
        "nodes": [
          {
            "timelogs": {
              "totalSpentTime": null,
              "nodes": [
                {
                  "id": null,
                  "mergeRequest": null,
                  "spentAt": null,
                  "summary": null,
                  "timeSpent": null,
                  "user": {
                    "username": null
                  }
                },
                {
                  "id": null,
                  "mergeRequest": null,
                  "spentAt": null,
                  "summary": null,
                  "timeSpent": null,
                  "user": {
                    "username": null
                  }
                },
                {
                  "id": null,
                  "mergeRequest": null,
                  "spentAt": null,
                  "summary": null,
                  "timeSpent": null,
                  "user": {
                    "username": null
                  }
                },
                {
                  "id": null,
                  "mergeRequest": null,
                  "spentAt": null,
                  "summary": null,
                  "timeSpent": null,
                  "user": {
                    "username": null
                  }
                }
              ],
              "pageInfo": {
                "endCursor": null,
                "startCursor": null,
                "hasNextPage": null,
                "hasPreviousPage": null
              }
            }
          }
        ],
        "pageInfo": {
          "endCursor": null,
          "startCursor": null,
          "hasNextPage": null,
          "hasPreviousPage": null
        }
      }
    }
  },
  "correlationId": null
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0