Sign inSign up

danchoi2001/strimzikafka

By danchoi2001

•Updated 7 months ago

just strimzi kafka with connector

Image
Message queues
0

1.1K

danchoi2001/strimzikafka repository overview

⁠Instruction

Download connectors to plugins folder. The structure of this folder look like this

.
├── Dockerfile
├── kafka-connect-build.yaml
├── kafka-connect.yaml
├── plugins
│   ├── debezium-debezium-connector-mongodb-2.4.2
│   ├── debezium-debezium-connector-postgresql-2.5.4
│   ├── mongodb-kafka-connect-mongodb-1.13.0
│   ├── scylladb-kafka-connect-scylladb-1.1.1
│   └── scylladb-scylla-cdc-source-connector-1.2.1
└── source-connector.yaml

Dockerfile:

FROM quay.io/strimzi/kafka:0.44.0-kafka-3.8.0
USER root
RUN mkdir -p /opt/kafka/plugins/
COPY ./plugins/ /opt/kafka/plugins/
USER 1001

Then build docker:

export DOCKER_ORG=danchoi2001
docker build . -t ${DOCKER_ORG}/strimzikafka
docker push ${DOCKER_ORG}/strimzikafka

⁠Release notes

⁠v0

.
├── Dockerfile
├── kafka-connect-build.yaml
├── kafka-connect.yaml
├── plugins
│   ├── debezium-debezium-connector-mongodb-2.4.2
│   ├── debezium-debezium-connector-postgresql-2.5.4
│   ├── mongodb-kafka-connect-mongodb-1.13.0
│   ├── scylladb-kafka-connect-scylladb-1.1.1
│   └── scylladb-scylla-cdc-source-connector-1.2.1
└── source-connector.yaml

⁠v1

add avro to use key/value converter

.
├── Dockerfile
├── kafka-connect-build.yaml
├── kafka-connect.yaml
├── plugins
│   ├── confluentinc-kafka-connect-avro-converter-7.7.1
│   ├── debezium-debezium-connector-mongodb-2.4.2
│   ├── debezium-debezium-connector-postgresql-2.5.4
│   ├── mongodb-kafka-connect-mongodb-1.13.0
│   ├── scylladb-kafka-connect-scylladb-1.1.1
│   └── scylladb-scylla-cdc-source-connector-1.2.1
└── source-connector.yaml

⁠v2->v4: just upgrade mongo kafka connect

⁠v5->v7: Build with multi stages and SMT -> Reduce image size

FROM confluentinc/cp-kafka-connect:7.8.0 AS cp
RUN confluent-hub install --no-prompt mongodb/kafka-connect-mongodb:1.14.1
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-avro-converter:7.8.0
RUN confluent-hub install --no-prompt debezium/debezium-connector-postgresql:2.5.4
RUN confluent-hub install --no-prompt debezium/debezium-connector-mongodb:2.4.2

FROM quay.io/strimzi/kafka:0.44.0-kafka-3.8.0
USER root:root
RUN mkdir -p /opt/kafka/plugins/custom
COPY --from=cp /usr/share/confluent-hub-components/mongodb-kafka-connect-mongodb/lib /opt/kafka/plugins/mongodb/
COPY --from=cp /usr/share/confluent-hub-components/confluentinc-kafka-connect-avro-converter/lib /opt/kafka/plugins/mongodb/
COPY --from=cp /usr/share/confluent-hub-components/debezium-debezium-connector-mongodb/lib /opt/kafka/plugins/mongodb/
COPY --from=cp /usr/share/confluent-hub-components/debezium-debezium-connector-postgresql/lib /opt/kafka/plugins/postgresql/

# Copy custom plugins
COPY plugins/ /opt/kafka/plugins/custom

USER 1001

Tag summary

Content type

Image

Digest

sha256:31c694bd6…

Size

372.6 MB

Last updated

7 months ago

docker pull danchoi2001/strimzikafka:v15.1