Skip to content

Support deploying syndicated views to stage #8532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 3, 2025
Merged

Support deploying syndicated views to stage #8532

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0ebe625
Support for deploying syndicated views to stage
scholtzan Dec 1, 2025
7ae15c7
Don't publish or replace references of syndicate views in stage deploys
scholtzan Dec 2, 2025
e25d639
Address review feedback
scholtzan Dec 2, 2025
62dd5e6
test app store view deploys
scholtzan Dec 2, 2025
c1dd11f
get schemas for tables that aren't in the repo
scholtzan Dec 2, 2025
f05d0aa
undo app store test
scholtzan Dec 2, 2025
7b8e7b1
Address review feedback
scholtzan Dec 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions bigquery_etl/cli/stage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -310,8 +310,18 @@ def _view_dependencies(artifact_files, sql_dir):
if not path.exists():
path.mkdir(parents=True, exist_ok=True)
# don't create schema for wildcard and metadata tables
# Create schemas for syndicated tables, stable, live tables and other
# tables not managed by bigquery-etl by doing a dryrun.
# The stage project doesn't have access to prod tables (e.g when referenced)
# so we need to create the schema here and deploy it.
if "*" not in name and name != "INFORMATION_SCHEMA":
partitioned_by = "submission_timestamp"
partitioned_by = None

if any(
dataset.endswith(suffix) for suffix in ("_live", "_stable")
):
partitioned_by = "submission_timestamp"

schema = Schema.for_table(
project=project,
dataset=dataset,
Expand All @@ -321,11 +331,11 @@ def _view_dependencies(artifact_files, sql_dir):
)
schema.to_yaml_file(path / SCHEMA_FILE)

if not file_exists_for_dependency:
(path / QUERY_SCRIPT).write_text(
"# Table stub generated by stage deploy"
)
view_dependencies.add(path / QUERY_SCRIPT)
if not file_exists_for_dependency:
(path / QUERY_SCRIPT).write_text(
"# Table stub generated by stage deploy"
)
view_dependencies.add(path / QUERY_SCRIPT)

# extract UDF references from view definition
raw_routines = read_routine_dir()
Expand Down
6 changes: 3 additions & 3 deletions bigquery_etl/view/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -426,9 +426,9 @@ def publish(self, target_project=None, dry_run=False, client=None):

if not self.metadata:
print(f"Missing metadata for {self.path}")

table.description = self.metadata.description
table.friendly_name = self.metadata.friendly_name
else:
table.description = self.metadata.description
table.friendly_name = self.metadata.friendly_name

if table.labels != self.labels:
labels = self.labels.copy()
Expand Down