docker?部署Flink和傳統(tǒng)部署的詳細(xì)過程

更新時間：2025年11月24日 12:06:01 作者：???傀儡師

Flink是一個開源的分布式流處理框架,支持狀態(tài)管理、事件時間、窗口模型等特性,可以通過下載、解壓、配置和啟動來使用,提供了多種運(yùn)行方式,包括Docker和SQLClient,本文介紹docker部署Flink和傳統(tǒng)部署的詳細(xì)過程,感興趣的朋友跟隨小編一起看看吧

簡單介紹

Flink 是一個開源的、分布式的、高性能的流處理框架，主要特性有：狀態(tài)管理與容錯，事件時間與水位線，先進(jìn)的窗口模型，輕量級的分布式快照，端到端的精確一次交付，流批一體與統(tǒng)一的 API，復(fù)雜的流處理拓?fù)渑c高級 API（側(cè)輸出，CEP：復(fù)雜事件處理庫，Table API & SQL）等
下載地址: https://mirrors.aliyun.com/apache/flink/ z根據(jù)需求選版本

下載安裝包

https://mirrors.aliyun.com/apache/flink/flink-1.20.3/flink-1.20.3-bin-scala_2.12.tgz

解壓

tra -zxf flink-1.20.3-bin-scala_2.12.tgz

修改配置文件

jobmanager.rpc.address: master
jobmanager.rpc.port: 6123
jobmanager.bind-host: 0.0.0.0
jobmanager.memory.process.size: 1600m
taskmanager.bind-host: 0.0.0.0
taskmanager.host: master
taskmanager.memory.process.size: 1728m
taskmanager.numberOfTaskSlots: 2
parallelism.default: 2
jobmanager.execution.failover-strategy: region
rest.address: localhost
rest.bind-address: 0.0.0.0

啟動

bin/start-cluster.sh

登錄首頁： http://192.168.115.102:8081

Docker Run

拉取鏡像

x86架構(gòu)
docker pull registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink
arm架構(gòu)
docker pull registry.cn-hangzhou.aliyuncs.com/qiluo-images/linux_arm64_flink

創(chuàng)建網(wǎng)絡(luò)

docker network create flink-network

啟動JobManager

docker run -d \
  --name=flink-jobmanager \
  --network=flink-network \
  -p 8081:8081 \
  -e FLINK_PROPERTIES="jobmanager.rpc.address: flink-jobmanager" \
  -v /data/flink:/opt/flink/data \
  registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink \
  jobmanager

啟動TaskManager

docker run -d \
  --name=flink-taskmanager \
  --network=flink-network \
  -e FLINK_PROPERTIES="jobmanager.rpc.address: flink-jobmanager" \
  -v /data/flink:/opt/flink/data \
  registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink \
  taskmanager

帶依賴的完整版本
docker-compose-full.yml

version: "3.8"
services:
  jobmanager:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink
    container_name: jobmanager
    ports:
      - "8081:8081"
      - "6123:6123"
    command: jobmanager
    environment:
      - FLINK_PROPERTIES=jobmanager.rpc.address: jobmanager
    volumes:
      - ./data/jobmanager:/opt/flink/data
      - ./logs/jobmanager:/opt/flink/log
      - ./apps:/opt/flink/apps
    networks:
      - flink-network
  taskmanager:
    image: flink:1.17.2
    container_name: taskmanager
    depends_on:
      - jobmanager
    command: taskmanager
    scale: 2
    environment:
      - FLINK_PROPERTIES=jobmanager.rpc.address: jobmanager
      - TASK_MANAGER_NUMBER_OF_TASK_SLOTS=4
    volumes:
      - ./data/taskmanager:/opt/flink/data
      - ./logs/taskmanager:/opt/flink/log
      - ./apps:/opt/flink/apps
    networks:
      - flink-network
  zookeeper:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/zookeeper
    container_name: zookeeper
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    ports:
      - "2181:2181"
    networks:
      - flink-network
  kafka:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/kafka
    container_name: kafka
    depends_on:
      - zookeeper
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
      - ALLOW_PLAINTEXT_LISTENER=yes
    ports:
      - "9092:9092"
    networks:
      - flink-network
networks:
  flink-network:
    driver: bridge
volumes:
  flink-data:

使用SQL Client的完整環(huán)境
docker-compose-sql.yml

version: "3.8"
services:
  jobmanager:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink
    container_name: flink-jobmanager
    ports:
      - "8081:8081"
      - "6123:6123"
    command: jobmanager
    environment:
      - FLINK_PROPERTIES=
          jobmanager.rpc.address: jobmanager
          taskmanager.numberOfTaskSlots: 4
          parallelism.default: 2
    volumes:
      - ./data/jobmanager:/opt/flink/data
      - ./logs/jobmanager:/opt/flink/log
    networks:
      - flink-network
  taskmanager:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink
    container_name: flink-taskmanager
    depends_on:
      - jobmanager
    command: taskmanager
    scale: 2
    environment:
      - FLINK_PROPERTIES=
          jobmanager.rpc.address: jobmanager
          taskmanager.numberOfTaskSlots: 4
          parallelism.default: 2
    volumes:
      - ./data/taskmanager:/opt/flink/data
      - ./logs/taskmanager:/opt/flink/log
    networks:
      - flink-network
  sql-client:
    image: registry.cn-hangzhou.aliyuncs.com/qiluo-images/flink
    container_name: flink-sql-client
    depends_on:
      - jobmanager
    command: |
      bash -c "
      echo 'Waiting for Flink cluster to be ready...'
      while ! nc -z jobmanager 8081; do
        sleep 1
      done
      echo 'Starting SQL Client...'
      ./bin/sql-client.sh"
    volumes:
      - ./sql-scripts:/opt/flink/sql-scripts
    networks:
      - flink-network
    tty: true
    stdin_open: true
networks:
  flink-network:
    driver: bridge

到此這篇關(guān)于docker 部署Flink和傳統(tǒng)部署的詳細(xì)過程的文章就介紹到這了,更多相關(guān)docker 部署Flink內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: