AIP-4117

External Account Credentials (Workload Identity Federation)

Using workload identity federation, your application can access Google Cloud resources from Amazon Web Services (AWS), Microsoft Azure or any identity provider that supports OpenID Connect (OIDC) or SAML 2.0.

Traditionally, applications running outside Google Cloud have used service account keys to access Google Cloud resources. Using identity federation, you can allow your workload to impersonate a service account. This lets you access Google Cloud resources directly, eliminating the maintenance and security burden associated with service account keys.

Note: Because this AIP describes guidance and requirements in a language-neutral way, it uses generic terminology which may be imprecise or inappropriate in certain languages or environments.

Guidance

This section describes the general guidance of supporting non-Google external credentials (AWS, Azure, OIDC and SAML IdPs, etc) as a means of authentication.

Prerequisite

In order to use workload identity federation to access Google cloud resources from non-Google cloud platforms, the following steps are needed to configure workload identity pools, providers, service account impersonation and generate the JSON configuration file to be used by the auth libraries.

• Configure Workload Identity Federation from AWS
• Configure Workload Identity Federation from Microsoft Azure
• Configure Workload Identity Federation from an OIDC identity provider
• Configure Workload Identity Federation from a SAML identity provider

Configuration File Generation and Usage

After workload identity federation is configured, the JSON configuration file should be generated. This sample shows how an AWS configuration file is generated:

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --output-file=$FILEPATH.json

The following values would need to be replaced:

• PROJECT_NUMBER: Project number of the project that contains the workload identity pool.
• POOL_ID: ID of the workload identity pool.
• PROVIDER_ID: ID of the workload identity pool provider.
• SERVICE_ACCOUNT_EMAIL: Email address of the service account to impersonate.
• FILEPATH: File to save configuration to.

If you are using AWS IMDSv2, an additional flag --enable-imdsv2 should be added to the gcloud iam workload-identity-pools create-cred-config command:

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --enable-imdsv2 \
    --output-file=$FILEPATH.json

If you wish to configure the service account access token lifetime, an additional flag --service-account-token-lifetime-seconds should be added to the gcloud iam workload-identity-pools create-cred-config command (this example uses an AWS configuration, but the token lifetime can be configured for all workload identity federation providers):

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --service-account-token-lifetime-seconds=$TOKEN_LIFETIME \
    --output-file=$FILEPATH.json

The service-account-token-lifetime-seconds flag is optional. If not provided, this defaults to one hour. The minimum allowed value is 600 (10 minutes) and the maximum allowed value is 43200 (12 hours). If a lifetime greater than one hour is required, the service account must be added as an allowed value in an Organization Policy that enforces the constraints/iam.allowServiceAccountCredentialLifetimeExtension constraint.

The external identities configuration file can be used with Application Default Credentials. In order to use external identities with Application Default Credentials, the full path to this file should be stored in the GOOGLE_APPLICATION_CREDENTIALS environment variable.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/config.json

The library can now automatically choose the right type of client and initialize credentials from the context provided in the configuration file:

import google.auth

credentials, project = google.auth.default()

External account credentials can also be initialized explicitly using the generated configuration file.

# Sample for Azure or OIDC/SAML providers.
import json

from google.auth import identity_pool

json_config_info = json.loads(function_to_get_json_config())
credentials = identity_pool.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
    ['https://www.googleapis.com/auth/cloud-platform'])

# Sample for AWS.
import json

from google.auth import aws

json_config_info = json.loads(function_to_get_json_config())
credentials = aws.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
    ['https://www.googleapis.com/auth/cloud-platform'])

Expected Behavior

The auth libraries should use the information in the JSON configuration file to retrieve the external credentials and exchange them for Google access tokens using the GCP Security Token Service (via the token exchange endpoint https://sts.googleapis.com/v1/token) and then impersonating a service account by calling the IamCredentials generateAccessToken API to access GCP resources.

All external account JSON files must share the following fields:

The auth libraries and applications must follow the steps below for all types of external account credentials:

• Check credential_source to determine the necessary logic to retrieve the external credential which should be used to construct the subject token to pass to the STS endpoint. This is covered in detail for every credential configuration below.
• Construct the STS request, based on rfc8693:
  • STS audience should be constructed using the audience field.
  • grant_type must be urn:ietf:params:oauth:grant-type:token-exchange
  • requested_token_type must be urn:ietf:params:oauth:token-type:access_token
  • subject_token_type is the subject_token_type field as described in the RFC.
  • subject_token is the retrieved external credentials. Check the subsequent sections on how this is retrieved in various environments.
  • scope: the list of space-delimited, case-insensitive OAuth scopes that specify the desired scopes of the requested security token in the context of the service or resource where the token should be used. If service account impersonation is used, the cloud platform or IAM scope should be passed to STS and then the customer provided scopes should be passed in the IamCredentials call to generateAccessToken.
  • The STS token exchange URL should be the token_url (e.g. https://sts.googleapis.com/v1/token).
• Send the STS token exchange request to get the Google access token and its expiration.
• If the service_account_impersonation_url is available, trigger service account impersonation flow by POSTing to that endpoint with the previously returned Google access token.
  • If this is not available, end the flow and just use the STS access token for authorization.
  • The list of scopes also need to be provided for this endpoint. The customer provided scopes should be used for this endpoint.
  • In order to access this API, the (Cloud platform https://www.googleapis.com/auth/cloud-platform or IAM scope https://www.googleapis.com/auth/iam) are required in the underlying access token.
  • The service account access token lifetime also needs to be provided for this endpoint. The value in service_account_impersonation.token_lifetime_seconds will be used if it was provided, otherwise it will default to 1 hour.

Determining the subject token in AWS

External account configuration JSON files should contain the following information in the credential_source object to facilitate retrieval of AWS credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

The JSON file for AWS configuration files should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
    "imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
  }
}

The auth libraries and applications must follow the steps below:

• Check credential_source for environment ID. If the environment ID is aws${version}, this should be an AWS native credential.
• Inspect the version in the environment ID. If this is a newer unexpected error, trigger an error that the auth library needs to be updated to handle this type of credentials.
• Validate the host for the url, regional_url and imdsv2_session_token_url fields if they are provided. The host should either be 169.254.169.254 or fd00:ec2::254.
• If imdsv2_session_token_url is available, then fetch session token from imdsv2_session_token_url.
• Check the environment variables in the following order (AWS_REGION and then the AWS_DEFAULT_REGION) to determine the AWS region. If found, skip using the AWS metadata server to determine this value.
• If the region environment variables are not provided, use the region_url to determine the current AWS region. The API returns the zone name, e.g. us-east-1d. The region should be determined by stripping the last character, e.g. us-east-1.
• Check the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and the optional AWS_SESSION_TOKEN for the AWS security credentials. If found, skip using the AWS metadata server to determine these values.
• If url is available and the security credentials environment variables are not provided:
  • Call url to retrieve the attached AWS IAM role name to the current instance.
  • Call url/$ROLE_NAME to get the access key, secret key and security token needed to sign the GetCallerIdentity request.

• Construct the AWS signed request (AWS Signature Version 4) using the GetCallerIdentity regional_cred_verification_url (with the region substituted). This should be serialized by formatting it as a url-encoded JSON and passed as the subject_token to STS endpoint. Here is a sample of the JSON format used:

  {
    "url": "https://sts.us-east-1.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
    "headers": [
      {
        "value": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
        "key": "x-goog-cloud-target-resource"
      },
      {
        "value": "20200228T225005Z",
        "key": "x-amz-date"
      },
      {
        "value": "AWS4-HMAC-SHA256 Credential=AKIASOZTBDV4D7ABCDEDF/20200228/us-east-1/sts/aws4_request, SignedHeaders=host;x-amz-date,Signature=abcedefdfedfd",
        "key": "Authorization"
      },
      {
        "value": "sts.us-east-1.amazonaws.com",
        "key": "host"
      },
      {
        "value": "IQoJb3JpZ2luX2VjEIz//////////wEaCXVzLWVh...",
        "key": "x-amz-security-token"
      }
    ],
    "method": "POST",
    "body": ""
  }
  

  For the AWS token, STS requires a special header x-goog-cloud-endpoint to recognize that the token is for a specific workload identity provider.

Determining the subject token in Microsoft Azure and URL-sourced credentials

External account configuration JSON files should contain the following information in the credential_source object to facilitate retrieval of Azure and other URL-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

The JSON file for URL-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "url": "http://localhost:5000/token",
    "format": {
      "type": "json",
      "subject_token_field_name": "id_token"
    }
  }
}

Azure configuration files are a type of OIDC URL-sourced credentials.

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "headers": {
      "Metadata": "True"
    },
    "url": "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
    "format": {
      "type": "json",
      "subject_token_field_name": "access_token"
    }
  }
}

The auth libraries and applications must follow the steps below:

• Check credential_source has a url field and no environment_id, otherwise skip the rest of the steps.
• An HTTP GET request should be sent to this local url while injecting the headers key/values (if provided in the configuration file) in the request header. The request should respond with the external credentials subject token to be passed to STS token endpoint.
• Before parsing the token, check the format field.
• If the format is not available, assume the external credential returned by the URL response is provided in plain text format.
• If available, check if the type is json
  • If json, check the subject_token_field_name. For Azure, this is set to access_token.
  • Parse the file as JSON and then retrieve the external credential from the field name based on the value of subject_token_field_name.

Determining the subject token in file-sourced credentials

External account configuration JSON files contain the following information in the credential_source object to facilitate retrieval of file-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

The JSON file for file-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/var/run/saml/assertion/token"
  }
}

The auth libraries and applications must follow the steps below:

• Check credential_source has a file field and no environment_id. If not, this is not a file-sourced credential and the proceeding steps do not apply.
• Get the external credential from the file location specified by the credential_source.file field.
• Before parsing the token, check the format field.
• If the format is not available, assume the external credential is provided in plain text format.
• If available, check if the type is json
  • If json, check the subject_token_field_name.
  • Parse the file as JSON and then retrieve the external credential from the field name based on the value of subject_token_field_name.

Determining the subject token in executable-sourced credentials

External account configuration JSON files contain the following information in the credential_source object to facilitate retrieval of executable-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

The JSON file for executable-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
  "token_url": "https://sts.googleapis.com/v1/token",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL@project.iam.gserviceaccount.com:generateAccessToken",
  "credential_source": {
    "executable": {
      "command": "/path/to/executable --arg1=value1 --arg2=value2",
      "timeout_millis": 5000,
      "output_file": "/path/to/cached/credentials"
    }
  }
}

To use executable-sourced credentials, the GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES environment variable must be set to 1.

Additionally, the executable must adhere to the following response format:

Successful responses:

A sample successful executable OIDC response:

{
  "version": 1,
  "success": true,
  "token_type": "urn:ietf:params:oauth:token-type:id_token",
  "id_token": "...",
  "expiration_time": 1620499962
}

A sample successful executable SAML response:

{
  "version": 1,
  "success": true,
  "token_type": "urn:ietf:params:oauth:token-type:saml2",
  "saml_response": "...",
  "expiration_time": 1620499962
}

Error responses:

A sample executable error response:

{
  "version": 1,
  "success": false,
  "code": "401",
  "message": "Caller not authorized."
}

The auth libraries and applications must follow the steps below:

• Check credential_source has an executable field and no environment_id. If not, this is not a executable-sourced credential and the proceeding steps do not apply.
• Retrieve the external credential's executable information from the credential_source.executable field.
• Check that the GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES environment variable is set to 1. If not, error out.
• Before the next step, check if credential_source.executable.output_file was specified in the credential configuration.
  • If present, check if there is an executable response at that location.
  • If the response is valid and unexpired, or there is no response at that location, continue execution.
  • If the response is malformed or invalid, error out.
• Ensure the following environment variables will be available to the executable:
  • GOOGLE_EXTERNAL_ACCOUNT_AUDIENCE: The audience field from the credential configuration. Must always be present.
  • GOOGLE_EXTERNAL_ACCOUNT_TOKEN_TYPE: The subject token type. Must always be present.
  • GOOGLE_EXTERNAL_ACCOUNT_IMPERSONATED_EMAIL: The service account email. Only present when service account impersonation is used.
  • GOOGLE_EXTERNAL_ACCOUNT_OUTPUT_FILE: The output file location from the credential configuration. Only present when specified in the credential configuration.
• Run the command specified at credential_source.executable.command.
  • Fail in the following scenarios:
    • The executable failed to complete in the timeout duration specified.
    • The executable's response is invalid, was unsuccessful or expired.
    • The executable finished with a non-zero exit code.
• Parse the executable response as JSON and then retrieve the external credential from the field name based on the value of token_type.
  • The token_type value must be urn:ietf:params:oauth:token-type:jwt, urn:ietf:params:oauth:token-type:id_token, or urn:ietf:params:oauth:token-type:saml2.
  • If the token_type is urn:ietf:params:oauth:token-type:saml2, the subject token will be parsed from the saml_response field.
  • Otherwise it will be parsed from the id_token field.

Changelog

• 2021-12-10: Add AIP for External Account Credentials (AIP 4117).
• 2022-05-18: Document executable-sourced credentials (AIP 4117).
• 2022-08-31: Document configurable token lifetime (AIP 4117).
• 2023-09-12: Mark region_url as optional (AIP 4117).