Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

retry test_big_query_write_temp_table_append_schema_update up to 3 times #31364

Merged
merged 6 commits into from
May 24, 2024
Merged

retry test_big_query_write_temp_table_append_schema_update up to 3 times #31364

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d63120d
added pytest.mark.flaky for test_big_query_write_temp_table_append_sc…
liferoad May 21, 2024
4bcf8ec
updated the version
liferoad May 21, 2024
092fe9b
only run flaky when it is available
liferoad May 21, 2024
6748273
Merge branch 'master' into bq-job-retry
liferoad May 24, 2024
fce07e8
switched to tenacity
liferoad May 24, 2024
a6a78e3
fixed the isort
liferoad May 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion sdks/python/apache_beam/io/gcp/bigquery_write_it_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@
from parameterized import param
from parameterized import parameterized

from tenacity import retry
from tenacity import stop_after_attempt

import apache_beam as beam
from apache_beam.io.gcp.bigquery import BigQueryWriteFn
from apache_beam.io.gcp.bigquery_tools import BigQueryWrapper
Expand Down Expand Up @@ -457,7 +460,7 @@ def test_big_query_write_insert_errors_reporting(self):
@pytest.mark.it_postcommit
def test_big_query_write_insert_non_transient_api_call_error(self):
"""
Test that non-transient GoogleAPICallError errors returned
Test that non-transient GoogleAPICallError errors returned
by beam.io.WriteToBigQuery are not retried and result in
FAILED_ROWS containing both the failed rows and the reason
for failure.
Expand Down Expand Up @@ -504,6 +507,7 @@ def test_big_query_write_insert_non_transient_api_call_error(self):
equal_to(bq_result_errors))

@pytest.mark.it_postcommit
@retry(reraise=True, stop=stop_after_attempt(3))
@parameterized.expand([
param(file_format=FileFormat.AVRO),
param(file_format=FileFormat.JSON),
Expand Down
Loading