Potential Service Account Leak when creating a roleset and bind with a resource it does not have permission

[lanluo2020]

If we try to create a roleset with the bindings which we don't have permission to it:
expected behavior:
failed to create the roleset, no service account is created on gcp

We found it does return with 403 Error:
"unable to get policy: googleapi: Error 403: The caller does not have permission"
However the service account is created on gcp and it is not cleaned up
even after run vault delete the roleset the service account still exists

The service account created here never rolled back(potential service account leaking)

vault-plugin-secrets-gcp/plugin/role_set.go

Line 153 in f90dbea

walId, err := rs.newServiceAccount(ctx, s, iamAdmin, project)

reproduce step:
1>you can create a roleset and bind with a bucket or gcp project id it does not have permission to it, or even bind with a non-exist bucket
2>run

vault write gcp/roleset/mytest-roleset \
project="your-gcp-project-id" \
secret_type="service_account_key" \
[email protected] 

you will get the 403 Error
3> go to gcp console, the service account is created regardless the error
4> vault read gcp/roleset/mytest-roleset
No value found at gcp/roleset/mytest-roleset

[sethvargo]

https://www.vaultproject.io/docs/secrets/gcp#roleset-creation-may-partially-fail

[sbunce]

(lanluo and I are coworkers)

That's a super-sharp edge-case. We're using the operator pattern with vault so we just continually overwrite the vault value to converge our rolesets to a specification. If we accidentally create a binding for a GCS bucket which doesn't exist, or if we remove a GCS bucket and don't update the binding, then all of our production GCP projects (hundreds of them) get filled up with leaked service accounts until they hit the project limit.

I'm trying to think about how we could work-around this in our operator. If anyone has run into this issue, and has a pattern for solving it, let me know.