What is it ?

It is Apache Flink with Ververica Postgres CDC built as one fat|uber jar to be launched via java -jar as a standalone application.

The purpose is - declaratively (via Flink SQL) replicate data from PostgreSQL to any other datastores.

A docker image of PostgreSQL 16 configured by Debuzium included to be run via docker-compose up -d.

Sample data is placed to the predefined by Postgres Docker docker-entrypoint-initdb.d directory.

Concrete example to replicate data to Elasticsearch 7 derived from example is in branch elastic.

Build

./gradlew clean shadowJar

Run

java -jar /home/nkonev/javaWorkspace/flink-pipe/build/libs/flink-pipe-0.1.0-all.jar

Links

• Streaming SQL with Apache Flink: A Gentle Introduction
• How-to guide: Build Streaming ETL for MySQL and Postgres based on Flink CDC
• Streaming ETL for MySQL and Postgres with Flink CDC
• Потоковый захват изменений из PostgreSQL/MySQL с помощью Apache Flink
• Как использовать Spring в качестве фреймворка для Flink-приложений
• debezium/container-images