Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the multiAddress.enabled cluster option present in zk >3.6.0 #595

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

cstaff14
Copy link

Change log description

  • Enable the multiaddreses.Enabled JVM flag in the zkserver.sh
  • parse inputted extra addresses and add them to the zoo config
    (2-3 concise points about the changes in this PR. When committing this PR, the committer is expected to copy the content of this section to the merge description box)

Purpose of the change

In zookeeper > 3.6.0 you can specify multiple addresses for a singular zookeeper server. here is the description of that cluster option. This update allows you to leverage that feature through the operator. This can also be useful if you want to expose individual zk servers through external addresses.
(e.g., Fixes #666, Closes #1234)

What the code does

  • Add funcitionality to parse a list of zk server addresses in additionalConfig when creating the config map
  • in the zookeeper image, parse those additional addresses and properly use them in the zk config
    • The operator currently writes server addresses on write config and node registration. When one of these events happen we add an extra step that:
      • parses the new addServerAddresses.txt created by the config map for addresses listed for the current server ID
      • if there is a "|" it formats each address using the zkconfig function, otherwise it adds the singular address
      • it also removes the client port that is appended by the zkconfig function for all addresses except the final one in the list. This is to follow formatting for the multiAddress feature as the client port should only be present once per list of servers

(Detailed description of the code changes)

How to verify it

  1. add the -Dzookeeper.multiAddress.enabled=true flag to the value of the SERVER_JVMFLAGS env variable in your pod definition.
pod:
  ...
  env:
    - name: SERVER_JVMFLAGS
      value: "-Dzookeeper.multiAddress.enabled=true"
  1. Then add your additional addresses to the config definition of your ZookeeperCluster CRD in the format server.<id>: <address1>|<address2>|...
    config:
      ...
      additionalConfig {
        server.1: "zoo1-net1.net|zoo1-net2.net",
        server.2: "zoo2-net1.net",
        server.4: "zoo4-net1.net|zoo4-net2.net|zoo4-net3.net"
      }
    
    
  2. Validate correct address list is displayed through k8s portforwarding
    $ kubectl port-forward -n default zookeeper-0 2181:2181
    
     $ echo conf | nc localhost 2181
        ...
        membership: 
        server.1=app-zookeeper-0.app-zookeeper-headless.namespace.svc.cluster.local:2888:3888:participant|zoo1-net1.net:2888:3888|zoo1-net2.net:2888:3888:participant;0.0.0.0:2181
        server.2=app-zookeeper-0.app-zookeeper-headless.namespace.svc.cluster.local:2888:3888:participant|zoo2-net1.net:2888:3888:participant;0.0.0.0:2181
        server.3=app-zookeeper-0.app-zookeeper-headless.namespace.svc.cluster.local:2888:3888:participant;0.0.0.0:2181
        server.4=app-zookeeper-0.app-zookeeper-headless.namespace.svc.cluster.local:2888:3888:participant|zoo4-net1.net:2888:3888:participant|zoo4-net2.net:2888:3888:participant|zoo4-net3.net:2888:3888:participant;0.0.0.0:2181
        server.5=app-zookeeper-0.app-zookeeper-headless.namespace.svc.cluster.local:2888:3888:participant;0.0.0.0:2181
    
    

amuraru and others added 12 commits November 17, 2023 19:39
* Make zookeeper ids start from an offset

* Bootstrap zookeeper cluster from an external SEED_NODE

* Use public ip for internal quorum election

* Delay zookeeper start to allow for LB registration

* Fix a bug when a node is teared down

* Implement review
- k8s libs to v0.27.5
- controller-runtime to v0.15.0

Updated usage for fake client - now required WithStatusSubresource(..)
MultiNamespacedCacheBuilder is deprecated switched to usage
Cache.Options.Namespaces.
Updated usage for depricated wait.Poll and use
fake.NewClientBuilder()

Fixes: pravega#567

Signed-off-by: Ann Taraday <[email protected]>

Change-Id: Ia673486d8ebb3c76e86d1cc3af846cd890b4ffaa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants