Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres CDC sync is making the confirmed flush LSN to a value > saved state LSN #55846

Open
1 task
vrishin-bolt opened this issue Mar 19, 2025 · 0 comments
Open
1 task
Labels

Comments

@vrishin-bolt
Copy link

Connector Name

source-postgres

Connector Version

v3.6.28

What step the error happened?

During the sync

Relevant information

Hi all,

Something super weird is happening -

  1. I am doing PG -> BQ, sync in CDC mode (im in latest PG source and BQ destination version and airbyte version 0.64.2)
  2. This is the first sync => its going to load everything

Confirmed flush LSN (2 mins before sync start)
-- airbyte_slot_2 C1/20E83D08 829480779016

Confirmed flush LSN (2 mins after sync start)
-- airbyte_slot_2 C1/22539BC0 829504592832

State LSN saved in airbyte state file
829504592784

⁉️ Clearly state LSN < confirmed flush LSN
Airbyte is saving an LSN > confirmed flush LSN when it starts? (or)
Airbyte is confirm that it has read and processed a LSN > what it has saved? Why??

This is causing my sync to fail with
"Saved offset is before replication slot's confirmed LSN" after sometime

I have attached the logs (I stopped the sync the moment I noticed this happened)

Airbyte Logs.txt

Relevant log output

Contribute

  • Yes, I want to contribute
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants