Skip to content

Commit

Permalink
Add a simple automation to generate new test credentials. (#1931)
Browse files Browse the repository at this point in the history
* Add a simple automation to generate new test credentials.

* Missing new line
  • Loading branch information
Ceridan authored Feb 21, 2025
1 parent 9abb313 commit 77ab53b
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 15 deletions.
9 changes: 0 additions & 9 deletions scripts/create_secret

This file was deleted.

41 changes: 41 additions & 0 deletions scripts/update_test_credentials
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/usr/bin/env bash

# Generates new test credentials.
#
# The script must be executed from the repository root folder.
#
# Dependencies:
# - gcloud
# - jq

SECRET_JSON_PATH=test_credentials/secret.json
BIGQUERY_JSON_PATH=test_credentials/bigquery.json
BIGQUERY_JSON_ENC_PATH=test_credentials/bigquery.json.enc

# Generate a new key for dataform-testing service account and download it.
gcloud iam service-accounts keys create "${SECRET_JSON_PATH}" \
--iam-account=dataform-testing@dataform-open-source.iam.gserviceaccount.com \
--project=dataform-open-source

# Create bigquery.json for encryption. Basically we do the same thing
# as `dataform init-creds` will do but without creating dataform project.
cat <<EOF > "${BIGQUERY_JSON_PATH}"
{
"projectId": "dataform-open-source",
"credentials": $(jq -Rsa < ${SECRET_JSON_PATH}),
"location": "US"
}
EOF

# Create encrypted secret
gcloud kms encrypt \
--ciphertext-file="${BIGQUERY_JSON_ENC_PATH}" \
--plaintext-file="${BIGQUERY_JSON_PATH}" \
--project=dataform-open-source \
--keyring=dataform-builder-keyring \
--key=dataform-builder-key \
--location=global

# Cleanup secrets
rm -f "${SECRET_JSON_PATH}"
rm -f "${BIGQUERY_JSON_PATH}"
7 changes: 1 addition & 6 deletions test_credentials/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,7 @@ package(default_visibility = ["//visibility:public"])

# You can update the credentials for testing by:
# * Ensuring you have the required permissions (at time of writing, you need to request a grant).
# * Go to the "Service Accounts" page on the "dataform-open-source" GCP project.
# * Expand the "[email protected]" account.
# * Go to the "Keys" tab, click "Add Key", and select "Create a New Key", in JSON format.
# * Download the key, and run the `dataform init-creds` CLI command on the JSON.
# * Copy the resulting `.df-credentials` file to here as `bigquery.json`.
# * Run the "scripts/create_secret" script.
# * Run the "scripts/update_test_credentials" script.
gcloud_secret(
name = "bigquery.json",
testonly = 1,
Expand Down

0 comments on commit 77ab53b

Please sign in to comment.