Description

When I change the fields under spec.config, zookeeper-operator does not issue a rolling update to roll out the changed config. For example, if I change the spec.config.commitLogCount field to 100, the operator reconciles the configMap to reflect the change. So in the pod, /conf/zoo.cfg which is where the configMap mounted has the commitLogCount field set to 100. But the /data/conf/zoo.cfg which is the config used by zookeeper still has the commitLogCount set to default value as 500.

Steps to reproduce:

1. Deploy a simple zookeeper cluster with the following yaml:

apiVersion: "zookeeper.pravega.io/v1beta1"
kind: "ZookeeperCluster"
metadata:
  name: "zookeeper"
spec:
  replicas: 3

2. Change the zookeeper yaml to configure the commitLogCount to 100

apiVersion: "zookeeper.pravega.io/v1beta1"
kind: "ZookeeperCluster"
metadata:
  name: "zookeeper"
spec:
  replicas: 3
  config:
    commitLogCount: 100

3. Observe that the pods didn't restart, and go into the pod and check the config used by zookeeper in /data/conf/zoo.cfg:

metricsProvider.exportJvmInfo=true
dataDir=/data
4lw.commands.whitelist=cons, envi, conf, crst, srvr, stat, mntr, ruok
syncLimit=2
commitLogCount=500
metricsProvider.httpPort=7000
snapSizeLimitInKb=4194304
standaloneEnabled=false
metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
initLimit=10
minSessionTimeout=4000
snapCount=10000
admin.serverPort=8080
autopurge.purgeInterval=1
maxSessionTimeout=40000
maxCnxns=0
globalOutstandingLimit=1000
reconfigEnabled=true
skipACL=yes
autopurge.snapRetainCount=3
tickTime=2000
quorumListenOnAllIPs=false
preAllocSize=65536
maxClientCnxns=60
dynamicConfigFile=/data/conf/zoo.cfg.dynamic.100000010
root@zookeeper-0:/data/conf# cat zoo.cfg | grep con
4lw.commands.whitelist=cons, envi, conf, crst, srvr, stat, mntr, ruok
reconfigEnabled=true
dynamicConfigFile=/data/conf/zoo.cfg.dynamic.100000010

Importance

should-have

Location

Zookeeper-operator is missing the functionality to restart the pods when config is changed.

Suggestions for an improvement

We suggest to attach the hash of config as annotations to zookeeper's statefulSet's template. So that when the config is changed, the changed annotation would trigger statefulSet's rolling update.