Skip to content

Remove the GetSubscription check since it requires extra subscriptions.get permission #383

@rQ-Qrr

Description

@rQ-Qrr
  • The CPS source connector calls GetSubscription to check for the existence + auth access to a subscription. The returned Subscription object is not used at all. This requires the user to grant the Pub/Sub Viewer role to the principal, in addition to the Pub/Sub Subscriber role.

  • Whether a subscription exists or not is already checked when we do the actually Pulls. Other client libraries (like the Python one) do not require this extra call. It's possible the Subscription could be useful for extra behavior - but here it's not used for anything.

  • The BQ sink connector does not check for BQ table existence upon startup. That's because it can be configured to create tables.

  • The GCS sink connector does not check for GCS bucket existence upon startup. It only creates blobs, not entire buckets. I expect this connector will fail upon first flush to the bucket, if the bucket does not exist.

  • The CPS sink connector does not check for CPS Topic existence upon startup. It creates a publisher at task startup. Task startup is separate from connector start up and happens afterwards, so I don't think the entire connector will necessarily fail to start up. I see a case for making the publisher at connector startup, so connector creation doesn't return success, only for task startup to ƒail later.

  • The Debezium Spanner source/CDC connector does not connect to Spanner upon startup. That happens at task startup.

Given that the problem of connector startup errors vs task startup errors is common in the ecosystem, we could just remove the GetSubscription check in the CPS source connector.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions