Zookeeper 安装 - docker-compose

本文是为了搭建zookeeper集群，并使用SASL认证。

文件

• docker-compose.yml

cat <<EOF > ./docker-compose.yml
version: "3"
services:
  zookeeper:
    image: zookeeper:3.6.3
   _name: zookeeper
    user: root
    restart: always
    ports:
      - 2181:2181
      - 2888:2888
      - 3888:3888
    environment:
      ZOO_MY_ID: 3
      TZ: Asia/Shanghai
    volumes:
      - ./conf/zoo.cfg:/conf/zoo.cfg
      - ./conf/zookeeper_server_jaas.conf:/conf/zookeeper_server_jaas.conf
      - ./conf/java.env:/conf/java.env
      - ./data/data:/data
      - ./data/datalog:/datalog
      - ./data/logs:/logs


EOF

• 三台机器，初ZOO_MY_ID外，其他配置都一样。
• node1：ZOO_MY_ID: 1 node2：ZOO_MY_ID: 2 node3：ZOO_MY_ID: 3
• zk启动的时候，会自动读取/conf/java.env文件，作为JVM的参数

• zoo.cfg

cat <<EOF > ./conf/zoo.cfg

dataDir=/data
dataLogDir=/datalog
tickTime=2000
initLimit=5
syncLimit=2
autopurge.snapRetainCount=3
autopurge.purgeInterval=0
maxClientCnxns=60
standaloneEnabled=true
admin.enableServer=true
quorumListenOnAllIPs=true

server.1=10.3.4.156:2888:3888;2181
server.2=10.3.4.157:2888:3888;2181
server.3=10.3.4.158:2888:3888;2181

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
sessionRequireClientSASLAuth=true
#requireClientAuthScheme=sasl
jaasLoginRenew=3600000

EOF

• 所有节点配置一样

• zookeeper_server_jaas.conf

cat <<EOF > ./conf/zookeeper_server_jaas.conf

Server {
       org.apache.zookeeper.server.auth.DigestLoginModule required
       user_admin="admin123"
       user_kafka="kafka123"
       ;
};

Client {
       org.apache.zookeeper.server.auth.DigestLoginModule required
       username="kafka"
       password="kafka123"
       ;
};

EOF

• 所有节点配置一样
• Server定义两个用户，admin是zookeeper节点之间交互使用的账号；kafka是为了未来让kafka连接zk使用的账号；
• Client定义的目的，是为了让zk Client Shell （zkCli.sh）能正常使用。后面会详细说明。

• java.env

cat <<EOF > ./conf/java.env
# 指定jaas文件的位置
SERVER_JVMFLAGS="-Djava.security.auth.login.config=/conf/zookeeper_server_jaas.conf"
EOF

• 所有节点配置一样

启动

docker-compose up -d

• 所有节点都启动

启动以后，我们通过zkServer.sh status查看集群状态

[root@devops-elk-all-zkf1 zookeeper]# docker exec -it zookeeper bash ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@devops-elk-all-zkf2 zookeeper]# docker exec -it zookeeper bash ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@devops-elk-all-zkf3 zookeeper]# docker exec -it zookeeper bash ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader

zkCli.sh With SASL

由于集群需要SASL认证，如果直接使用zkCli.sh进去后，无法执行命令（如ls /，会直接报授权失败）：

[root@devops-elk-all-zkf1 zookeeper]# docker exec -it zookeeper bash ./bin/zkCli.sh
Connecting to localhost:2181

......

[zk: localhost:2181(CONNECTED) 0] ls /
2023-03-17 15:18:28,822 [myid:localhost:2181] - WARN  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1300] - Session 0x1004e8f02a00000 for sever localhost/127.0.0.1:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException.
EndOfStreamException: Unable to read additional data from server sessionid 0x1004e8f02a00000, likely server has closed socket
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
KeeperErrorCode = Session closed because client failed to authenticate for /
[zk: localhost:2181(CONNECTED) 1]

如果要想使用zkCli.sh，必须需要配置client jaas文件。安装时，我们把client jaas和 server jaas 放在了同一个位置：/conf/zookeeper_server_jaas.conf
有两种方式可以让zkCli.sh使用SASL认证：

• 方式一

# 先进入到容器内
docker exec -it zookeeper bash
# 导出环境变量，指定zookeeper_server_jaas.conf位置
export JVMFLAGS="-Djava.security.auth.login.config=/conf/zookeeper_server_jaas.conf"
# 进入交互环境
./bin/zkCli.sh

......

[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]

• 方式一

# 进入容器时指定环境变量
docker exec -it -e JVMFLAGS="-Djava.security.auth.login.config=/conf/zookeeper_server_jaas.conf" zookeeper bash ./bin/zkCli.sh

......

[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]