Configuring an External ZooKeeper Ensemble

ZooKeeper is a framework used to centralized service for maintaining configuration information, mechanisms for fail-overs, and state management. In our setup, ZooKeeper will manage the distribution of configuration files across the cluster and keep each Solr server synchronized with each other.

You can download ZooKeeper from the links provided here. To know which version of ZooKeeper you should be using with your SolrCloud cluster, go to your Solr's <solr-home>/server/solr-webapp/webapp/WEB-INF/lib/ directory. In there, you should see the ZooKeeper library and it should tell you which version of ZooKeeper is compatible with your SolrCloud cluster.

Assumptions

• In this guide, we will be using Solr 6.2.1 and ZooKeeper 3.4.6.
• We will store our universal configurations in /datastore/apps/zookeeper/configs.

• We will configure three ZooKeeper instances namely zk1, zk2, and zk3.

  • zk1

    • Home directory: /datastore/apps/zookeeper/instances/zk1/
    • IP address: 192.168.21.71

  • zk2

    • Home directory: /datastore/apps/zookeeper/instances/zk2/
    • IP address: 192.168.21.72

  • zk3

    • Home directory: /datastore/apps/zookeeper/instances/zk3/
    • IP address: 192.168.21.73

Procedure

1. Download a copy of ZooKeeper's portable installer (zookeeper-3.4.6.tar.gz in our case), extract the file, and copy the extracted directory to each of your ZooKeeper instances' home directory.

   cd /datastore/apps/zookeeper/
   
   # Dowload, extract, and rename the installer.
   wget http://mirror.rise.ph/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
   tar -xvf zookeeper-3.4.6.tar.gz
   mv zookeeper-3.4.6 zookeeper
   
   # Copy the directory to each instance's home folder.
   cp -r zookeeper instances/zk1/
   cp -r zookeeper instances/zk2/
   cp -r zookeeper instances/zk3/
   

2. Create /data/<zookeper-id>/myid files in each instance's home directory where:

   • <zookeper-id> should be replaced with the ZooKeeper instance's respective ID number and;
   • Every myid file's content is also the ZooKeeper instance's respective ID number.

   # Create the directories first.
   mkdir -p instances/zk1/data/1
   mkdir -p instances/zk2/data/2
   mkdir -p instances/zk3/data/3
   
   # Create the files.
   echo "1" > instances/zk1/data/1/myid
   echo "2" > instances/zk2/data/2/myid
   echo "3" > instances/zk3/data/3/myid
   

3. After creating the data directories, we should now create each ZooKeeper instance's configuration file (zoo.cfg).

   # Open the configuration file.
   vi <zookeeper-home>/zookeeper/conf/zoo.cfg
   
   # Define the configuration we want.
   tickTime=2000
   initTime=10
   initLimit=5
   syncLimit=5
   clientPort=2181
   dataDir=<zookeeper-home>/data/<zookeper-id>
   server.1=192.168.21.71:2888:3888
   server.2=192.168.21.72:2888:3888
   server.3=192.168.21.73:2888:3888
   

   where:

   • <zookeeper-home> should be substituted with the ZooKeeper instance's home directory.
   • <zookeeper-id> should be substituted with the ZooKeeper instance's ID.

   Perform these commands for every ZooKeeper instance.

4. Lastly, start ZooKeeper by calling zkServer.sh start on each server.

   /datastore/apps/zookeeper/instances/zk1/zookeeper/bin/zkServer.sh start
   /datastore/apps/zookeeper/instances/zk2/zookeeper/bin/zkServer.sh start
   /datastore/apps/zookeeper/instances/zk3/zookeeper/bin/zkServer.sh start
   

That's it! Your ZooKeeper quorum is now ready with three instances up and running ready to serve Solr.