BigQuery LoadJobConfiguration doesn't respect destination dataset projectId #4482
Description
- OS: MacOS Mojave
- Java version: 9.0.1
- google-cloud-java version(s): 1.62.0
Steps to reproduce
gcloud auth application-default login using an account from some ProjectA (some-user@gsuite.domain in the code below). That account has BigQuery admin role in ProjectB and no BigQuery permissions in ProjectA.
Running the code below where ProjectB is some arbitrary GCP project different from the above ProjectA:
BigQuery bigQuery = BigQueryOptions.getDefaultInstance().getService();
TableId destinationTableId = TableId.of("ProjectB", "test_dataset", "test_table");
String sourceUri = "gs://some/test/bucket/data.json";
LoadJobConfiguration loadJobConfiguration =
LoadJobConfiguration
.builder(destinationTableId, sourceUri)
.setFormatOptions(FormatOptions.json())
.setAutodetect(true)
.build();
JobInfo jobInfo = JobInfo.of(loadJobConfiguration);
Job job = bigQuery.create(jobInfo);
job.waitFor();
results in the exception included below as the project from the default application credentials is used instead of the one specified for destination table.
Stacktrace
Exception in thread "main" com.google.cloud.bigquery.BigQueryException: Access Denied: Project ProjectA: The user some-user@gsuite.domain does not have bigquery.jobs.create permission in project ProjectA.
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:100)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.create(HttpBigQueryRpc.java:185)
at com.google.cloud.bigquery.BigQueryImpl$4.call(BigQueryImpl.java:241)
at com.google.cloud.bigquery.BigQueryImpl$4.call(BigQueryImpl.java:238)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:237)
at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:212)
at oc_perf_metrics.Test.main(Test.java:24)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Access Denied: Project ProjectA: The user some-user@gsuite.domain does not have bigquery.jobs.create permission in project ProjectA.",
"reason" : "accessDenied"
} ],
"message" : "Access Denied: Project ProjectA: The user some-user@gsuite.domain does not have bigquery.jobs.create permission in project ProjectA."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:401)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1097)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:499)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.create(HttpBigQueryRpc.java:183)
... 8 more
External references such as API reference guides used
I think there was a similar issue #3924 reported recently that got fixed in #4183 but it still doesn't work for me.
Workaround
Instead of using a default instance of BigQuery that would use ProjectA from the default credentials:
BigQuery bigQuery = BigQueryOptions.getDefaultInstance().getService();
use an instance that sets the default projectID to ProjectB explicitly:
BigQuery bigQuery = BigQueryOptions.newBuilder().setProjectId("ProjectB").build().getService();
Thanks!