Skip to content

Write a guide for setting up a pipeline (postgres -> kafka) #1855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lovromazgon opened this issue Sep 16, 2024 · 1 comment
Open

Write a guide for setting up a pipeline (postgres -> kafka) #1855

lovromazgon opened this issue Sep 16, 2024 · 1 comment
Labels
documentation Improvements or additions to documentation

Comments

@lovromazgon
Copy link
Member

lovromazgon commented Sep 16, 2024

This one is a bit more advanced than #1854, it already requires some setup (running Postgres and Kafka), creating some tables and data in the source and then consuming the data from Kafka to prove it reached the destination.

If possible, try to find a docker container for Postgres that already has some data included, or possibly has a table that is continuously getting data inserted/updated by an agent, or something.

@lovromazgon lovromazgon converted this from a draft issue Sep 16, 2024
@hariso hariso added the documentation Improvements or additions to documentation label Sep 18, 2024
@raulb
Copy link
Member

raulb commented Oct 4, 2024

We can probably rely on @hariso 's https://github.com/hariso/docker-containers/tree/main/postgresql and add some inserts, etc.

@lovromazgon lovromazgon removed their assignment Oct 7, 2024
@raulb raulb moved this from Todo to Backlog in Conduit Main Feb 6, 2025
@raulb raulb moved this from Backlog to Todo in Conduit Main Feb 6, 2025
@raulb raulb added this to the Next milestone Feb 6, 2025
@raulb raulb removed the status in Conduit Main Feb 8, 2025
@raulb raulb moved this to Todo in Conduit Main Feb 8, 2025
@raulb raulb removed this from the Next milestone Feb 8, 2025
@raulb raulb removed the status in Conduit Main Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
Status: No status
Development

No branches or pull requests

3 participants