快捷導(dǎo)航

Docker部署Kafka消息隊(duì)列系統(tǒng)完整指南

更新時(shí)間：2025年07月07日 08:24:47 作者：huingymm

在Docker中設(shè)置Kafka集群是一個(gè)相對(duì)簡(jiǎn)單的過(guò)程,因?yàn)樗试S你快速地啟動(dòng)和配置多個(gè)Kafka實(shí)例,這篇文章主要介紹了Docker部署Kafka消息隊(duì)列系統(tǒng)的相關(guān)資料,文中通過(guò)代碼介紹的非常詳細(xì),需要的朋友可以參考下

本指南將詳細(xì)介紹如何使用 Docker 部署 Kafka 消息隊(duì)列系統(tǒng)，包括單節(jié)點(diǎn)和集群模式的部署方式。

1. 單節(jié)點(diǎn)部署 (Zookeeper + Kafka)

1.1 創(chuàng)建 docker-compose.yml 文件

version: '3.8'

services:
  zookeeper:
    image: bitnami/zookeeper:3.8
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    volumes:
      - zookeeper_data:/bitnami/zookeeper

  kafka:
    image: bitnami/kafka:3.4
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://${HOST_IP}:9092
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    volumes:
      - kafka_data:/bitnami/kafka
    depends_on:
      - zookeeper

volumes:
  zookeeper_data:
    driver: local
  kafka_data:
    driver: local

1.2 啟動(dòng)服務(wù)

export HOST_IP=$(hostname -I | awk '{print $1}')
docker-compose up -d

2. KRaft 模式部署 (無(wú) Zookeeper)

2.1 創(chuàng)建 docker-compose.yml 文件

version: '3.8'

services:
  kafka:
    image: bitnami/kafka:3.4
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      - KAFKA_CFG_PROCESS_ROLES=controller,broker
      - KAFKA_CFG_NODE_ID=1
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@kafka:9093
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://${HOST_IP}:9092
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    volumes:
      - kafka_data:/bitnami/kafka

volumes:
  kafka_data:
    driver: local

2.2 啟動(dòng)服務(wù)

export HOST_IP=$(hostname -I | awk '{print $1}')
docker-compose up -d

3. 集群部署 (3節(jié)點(diǎn))

3.1 創(chuàng)建 docker-compose.yml 文件

version: '3.8'

services:
  zookeeper:
    image: bitnami/zookeeper:3.8
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOO_SERVER_ID=1
      - ZOO_SERVERS=0.0.0.0:2888:3888
    volumes:
      - zookeeper_data:/bitnami/zookeeper

  kafka1:
    image: bitnami/kafka:3.4
    container_name: kafka1
    ports:
      - "9092:9092"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_BROKER_ID=1
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://${HOST_IP}:9092
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    volumes:
      - kafka1_data:/bitnami/kafka
    depends_on:
      - zookeeper

  kafka2:
    image: bitnami/kafka:3.4
    container_name: kafka2
    ports:
      - "9093:9093"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_BROKER_ID=2
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://${HOST_IP}:9093
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    volumes:
      - kafka2_data:/bitnami/kafka
    depends_on:
      - zookeeper

  kafka3:
    image: bitnami/kafka:3.4
    container_name: kafka3
    ports:
      - "9094:9094"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_BROKER_ID=3
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://${HOST_IP}:9094
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    volumes:
      - kafka3_data:/bitnami/kafka
    depends_on:
      - zookeeper

volumes:
  zookeeper_data:
    driver: local
  kafka1_data:
    driver: local
  kafka2_data:
    driver: local
  kafka3_data:
    driver: local

3.2 啟動(dòng)集群

export HOST_IP=$(hostname -I | awk '{print $1}')
docker-compose up -d

4. 基本操作驗(yàn)證

4.1 創(chuàng)建主題

docker exec -it kafka kafka-topics.sh --create \
  --bootstrap-server localhost:9092 \
  --replication-factor 1 \
  --partitions 3 \
  --topic test-topic

4.2 生產(chǎn)消息

docker exec -it kafka kafka-console-producer.sh \
  --bootstrap-server localhost:9092 \
  --topic test-topic

4.3 消費(fèi)消息

docker exec -it kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic test-topic \
  --from-beginning

4.4 查看主題列表

docker exec -it kafka kafka-topics.sh --list \
  --bootstrap-server localhost:9092

5. 管理技巧

5.1 數(shù)據(jù)持久化

所有數(shù)據(jù)卷都配置在 volumes 部分，確保數(shù)據(jù)不會(huì)因容器重啟而丟失。

5.2 監(jiān)控配置

docker stats kafka zookeeper

5.3 日志查看

docker logs -f kafka

5.4 停止和清理

docker-compose down
# 如需刪除數(shù)據(jù)卷
docker-compose down -v

6. 注意事項(xiàng)

生產(chǎn)環(huán)境建議使用 KRaft 模式或至少 3 個(gè)節(jié)點(diǎn)的集群
根據(jù)實(shí)際需求調(diào)整 KAFKA_CFG_ADVERTISED_LISTENERS
數(shù)據(jù)卷路徑可根據(jù)需要修改
內(nèi)存限制可通過(guò) -m 參數(shù)設(shè)置
安全配置建議添加認(rèn)證機(jī)制

通過(guò)以上配置，您可以快速部署適用于開(kāi)發(fā)和測(cè)試環(huán)境的 Kafka 服務(wù)。生產(chǎn)環(huán)境請(qǐng)根據(jù)實(shí)際需求調(diào)整配置參數(shù)和安全設(shè)置。

到此這篇關(guān)于Docker部署Kafka消息隊(duì)列系統(tǒng)的文章就介紹到這了,更多相關(guān)Docker部署Kafka內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章:

最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线