docker-kafka部署

发表于 | 分类于 |

在此记录使用 docker-compose 启动一个 kafka 集群的命令

docker-kafka 部署

docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 10.0.1.149
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  kafka-manager:
    image: sheepkiller/kafka-manager                ## 镜像：开源的web管理kafka集群的界面
    environment:
        ZK_HOSTS: 10.0.1.149:2181                   ## 修改:宿主机IP
    ports:
      - "9000:9000"                                 ## 暴露端口

启动多个kafka 节点，比如3个

1
docker-compose scale kafka=3

或者启动的时候直接就启动 3 个节点

1
docker-compose up --scale kafka=3 -d

测试连通性

1
docker exec -it kafka_kafka_1 /bin/bash
  • 创建 topic
1
$KAFKA_HOME/bin/kafka-topics.sh --create --topic test --zookeeper kafka_zookeeper_1:2181 --replication-factor 1 --partitions 1
  • 查看 topic
1
$KAFKA_HOME/bin/kafka-topics.sh --zookeeper kafka_zookeeper_1:2181 --describe --topic test
  • 启动 producer
1
$KAFKA_HOME/bin/kafka-console-producer.sh --topic=test --broker-list kafka_kafka_1:9092
  • 启动 consumer
1
$KAFKA_HOME/bin/kafka-console-consumer.sh --bootstrap-server kafka_kafka_1:9092 --from-beginning --topic test
  • 查看消费组信息
1
kafka-consumer-groups.sh --bootstrap-server kafka_kafka_1:9092 --group test-group --describe
  • 重置消费族消费偏移量 
    1
    kafka-consumer-groups.sh --bootstrap-server kafka_kafka_1:9092 --group test-group --reset-offsets --topic test --to-earliest --execute