Create three node open cluster
This section illustrates the manual steps to form 3 node open cluster using the Syncfusion Data Integration Platform without security (User Management Server or AD configured).
Note: This document does not cover migrating from standalone installation to cluster.
- Machine setup
- Configure Zookeeper
- Changes in NiFi properties file.
- Delete flow.xml.gz file
- Start Data Integration Service
Step 1: Machine setup
Consider three machines with the following hostnames:
synclapn3382 (node 1)
syncdeskn2558 (node 2)
synclapn12786 (node 3)
Install the Syncfusion Data Integration Platform without User Management Server in all the three machines. Make sure that the DNS is resolved in all three nodes that is each machine should be able to ping other machines.
To install and configure the Data Integration Platform without User Management Server, refer here.
Step 2: Configure Zookeeper
Update zookeeper.properties file
To set up Zookeeper, edit “C:\Syncfusion\DataIntegration\220.127.116.11\SDK\NIFI\conf\zookeeper.properties”and add the following properties in all 3 nodes at end of the file
The actual hostnames and zookeeper ports (2888 & 3888) of each server is referred. To know more about Zookeeper configuration, refer here.
Note: The numbers server. (1, 2, 3) represents the myid values read from the myid file in the zookeeper directory to identify the server.
After editing, the properties file should look as follows:
Create “myid” file
Create a text file myid in all three nodes with a unique ID number to represent the server.
1.Create file named myid in all the 3 nodes under
2.Set the value in myid file as 1 in node1, 2 in node2, and 3 in node3.
Note: Create missing directory if any and create the myid without file extension.
Note: All the nodes should not have the same value in myid file. Node 1 should have a value as 1, Node 2 should have a value as 2 and goes on.
Step 3: Configure changes in NiFi properties
Set the following property values in “C:\Syncfusion\DataIntegration\18.104.22.168\SDK\NIFI\conf\nifi.properties” in all the nodes as follows:
Note: The property values of node 1 is synclapn3382, follow the similar steps in nodes 2 and 3.
|nifi.zookeeper.connect.string||synclapn3382:2181, syncdeskn2558:2181, synclapn12786:2181||Set proper machine name as per your machine setup.|
|nifi.web.http.host||synclapn3382||Set the machine name with its appropriate hostname of the cluster node. For node 2 – syncdeskn2558 and node 3 - synclapn12786|
|nifi.cluster.node.address||synclapn3382||Set the machine name with its appropriate hostname of the cluster node. For node 2 – syncdeskn2558 and node 3 - synclapn12786|
|nifi.cluster.node.protocol.port||7474||Any port that should not use by other process.|
Step 4: Delete flow.xml.gz file
Delete the file in the location “C:\Syncfusion\DataIntegration\22.214.171.124\SDK\NIFI\conf\flow.xml.gz” in all 3 nodes if they exist. These files will be auto generated based on your configuration files.
Step 5: Start Data Integration Service
1.Restart your Data Integration service through the Data Integration Service Manager.
2.For the first time alone, you will see that the cluster is in process of voting on the appropriate Data Flow as given in the following screenshot. It will take few minutes to take into the application canvas.
3.Once the application is launched, you can find number of connected nodes in the left side of the status bar and cluster tab under menu option.
Now, you will find the cluster formed successfully with three nodes.