Novell Home

AppNote: BorderManager Clustering with Load Balancing

Novell Cool Solutions: AppNote
By Steve Aitken

Digg This - Slashdot This

Posted: 25 Mar 2005
 

BorderManager Clustering with Load Balancing

by Steven Aitken

Introduction

This AppNote describes how to set up a two-node BorderManager web proxy cluster, with load balancing enabled between both nodes. This can be useful in high-demand or high- resilience solutions, as you will not only be able to load balance between the two servers, but you will be able to use two different ISPs if you so desire.

Intended Audience

This AppNote is intended for administrators who have current experience with NBM 3.8 and Novell Clustering technologies, and who intend to create a high-performance and resilient proxy solution.

You will require the following things to set up this example:

  • Two Netware 6.5 servers configured in the same tree
  • BorderManager 3.8 Proxy services set up on both servers
  • A working knowledge of Netware 6.5
  • A working knowledge of BorderManager 3.8
  • A working knowledge of Novell Clustering

Design Challenges

With the current architecture of proxy.nlm, several design challenges must be overcome to successfully enable it to be used in an active / active cluster role.

  • Proxy.nlm will not bind to an IP address without that address being entered in NWAdmin and flagged as a private IP.
  • Proxy.nlm will bind each IP address flagged as private as a secondary IP address on the server as it loads.
  • By default, if proxy.nlm detects a duplicate IP address on the network as it loads, it will not bind to that address.

To overcome these design difficulties, you need to perform the following steps:

Enter each IP address that your active / active proxy cluster listens on in NWAdmin and flag it as a private address on all cluster nodes.

Allow IP address duplicates on the cluster servers, so when proxy.nlm loads, it will successfully bind to all private addresses.

As soon as proxy.nlm finishes initializing, delete all secondary IP addresses manually to prevent network communication problems. Then you can use Novell Clustering to move these secondary addresses between nodes when required.

Caution: If you are running the active / active proxy cluster, and you reload proxy.nlm on one node, there will be a period of perhaps 10 seconds where all proxy listen addresses will be duplicated on your network. As this will cause brief disruption to internet services, you should not reboot a failed cluster node during production hours.

Deployment Example

We will use two Netware 6.5 servers - Server1 and Server2 - each with a separate Internet connection. These servers will be configured with IP addresses of 10.10.10.1 and 10.10.10.2. Clustered BorderManager proxy services will be available on IP addresses 10.10.10.10 and 10.10.10.11.

As we are not concerned with sharing cache volume information between cluster nodes, clustering BorderManager does not require any shared SAN or iSCSI storage - only network connectivity between the two nodes.

The proxy load balancing algorithm used will ensure that each of the caches stores independent and non-replicated data to ensure high hot-fill rates. This, however, may not be suitable for some users who authenticate web sessions by IP address or who do not use Clntrust for authentication, as each URL retrieved will be filled from a different proxy server. If you want to ensure your users retain a proxy server for their entire browsing session, please use the DNS round-robin method of client configuration rather than the proxy.pac method.

Installation

Before you begin, make sure both servers are running Netware 6.5 and have BorderManager 3.8 proxy services set up. If you need assistance during the build process, please review the Novell documentation at the links below:

Netware 6.5 Documentation:
http://www.novell.com/documentation/nw65/index.html

BorderManager 3.8 Documentation
http://www.novell.com/documentation/nbm38/index.html

Novell Clustering Documentation http://www.novell.com/documentation/ncs65/index.html

Step 1 - Configure Proxy listener addresses

Because the BorderManager proxy will only listen on IP addresses marked as private in NWADMIN, you must configure the proxy setup on both servers to listen to both the 10.10.10.10 and 10.10.10.11 addresses.

1. Load NWADMIN from one of the BorderManager servers and select the details page of Server1.

2. Navigate to the BorderManager Setup page and click the IP Addresses button.

3. Click New Address in the top right of the dialog and input the 10.10.10.10 address with the corresponding subnet mask.

Figure 1: Configuring the first IP address

4. Repeat this process for the 10.10.10.11 address, as shown below.

Figure 2: Configuring the second IP address

5. Repeat the above configuration for the server2 eDirectory object.

Step 2 - Create the proxy load script

You must create a new script to load the proxy on both nodes. Using EDIT.NLM, create a new file called LOADPXY.NCF with the following text and save into the SYS:\System directory of both nodes, as follows:

SET ALLOW IP ADDRESS DUPLICATES=ON
LOAD PROXY
DELETE SECONDARY IPADDRESS 10.10.10.10
DELEETE SECONDARY IPADDRESS 10.10.10.11

Step 3 - Setup Cluster

Install Cluster services onto both nodes as per the Novell documentation. You do not need to tick the shared storage option here.

Step 4 - Configure Cluster resources

You must now create two scripts that will allow for failover of the proxy addresses when the node fails.

1. Load ConsoleOne from one of your newly created cluster servers and highlight the cluster object in the left hand pane.

2. From the file menu, select New -> Object -> NCS:Cluster Resource.

3. Name the resource BMProxy1 and select to define additional properties.

4. Enter the following into the Cluster Load Script section:

SET ALLOW IP ADDRESS DUPLICATES = ON
ADD SECONDARY IPADDRESS 10.10.10.10

Figure 3: Proxy properties - Cluster Load Script

5. Enter the following into the Cluster Unload Script section:

DELETE SECONDARY IPADDRESS 10.10.10.10

Figure 4: Proxy properties - Cluster Unload Script

6. Repeat the above process, naming the second cluster resource BMProxy2 and substituting IP address 10.10.10.11

Step 5 - Assign proxies to nodes 1 & 2

Double-click the BMProxy1 resource and select the NODES tab. Ensure that both nodes are in the assigned column and that SERVER1 is at the top of the list, as shown below.

Figure 5: BMProxy1 resource with SERVER1 at top

Double-click the BMProxy2 resource and select the NODES tab. Ensure that both nodes are in the assigned column and that SERVER2 is at the top of the list.

Figure 6: BMProxy1 resource with SERVER2 at top

Step 6 - Edit servers Autoexec.ncf

There will be an entry at the end of the file to load "STARTBRD". This must be replaced with the "LOADPXY.NCF" you created in step 4. It is important that this line is BEFORE the LDNCS line to start the cluster.

Figure 7: Loading STARTBRD

Step 7 - Reboot both servers

After each server has successfully rebooted, the sequence of events will be as follows:

  1. BorderManager Proxy starts.
  2. Proxy.nlm automatically binds to 10.10.10.10 address.
  3. Proxy.nlm automatically binds to 10.10.10.11 address.
  4. Startpxy.ncf deletes both 10.10.10.10 and 10.10.10.11 secondary IP addresses.
  5. Clustering software loads.
  6. BMProxy1 resource starts on Server1 with IP address 10.10.10.10
  7. BMProxy2 resource starts on Server2 with IP address 10.10.10.11

Should either cluster node fail, the clustering software will migrate the 10.10.10.10 address to the other node. Because the proxy is configured to listen to this address, the user's Internet session will proceed as normal.

Step 8 - Preliminary Client Testing

Ensure that both proxies work as intended by reconfiguring your web browser to use the proxies at 10.10.10.10 and 10.10.10.11, port 8080.

Figure 8: Testing the proxies

Client Configuration

The clients must be configured to use cither the URLhash load balancing algorithm, or the DNS round-robin method to split the load between the two proxy servers. It will depend entirely on your network design and requirements as to which method will be most suitable.

Client Configuration with URL-based Load Balancing

You can split the load on both BorderManager servers by utilizing a URL hashing algorithm developed by Sharp. This will effectively re-route client requests to both proxy servers, depending on the URL requested. Using this mechanism, each BorderManager server's cache will be utilized most efficiently due to the fact that every client that requests the same URL will be directed to the same proxy server.

This method is suitable if you use NBM proxy services in any of the following ways:

  • CLNTRUST.EXE authenticates workstations to NBM.
  • An open system is run without access rules enabled.

This method is not suitable if you use NBM proxy services with

  • SSL-based authentication
  • Web applications that require IP address based security. For example, the URL https://securesite.domain.com/page1.html will use a different proxy from https://securesite.domain.com/page2.html, so the originating IP address will be different.

Example 1: Load Balanced proxy.pac Cluster

The UrlHash algorithm splits requests to the proxy, depending on URL requested.

Figure 9: Load Balanced proxy.pac cluster

Example 2: Failed Node

Should one Node fail, services will continue as normal.

Figure 10: Continued services with a failed node

Proxy.pac Details

In Notepad, create a text file called proxy.pac containing the following text:

/* Super Proxy Script 
   Copyright 1996 SHARP Corp.
   See http://naragw.sharp.co.jp/sps/

   Modified for 2 node cluster with
   50 / 50 load split
   For Novell CoolSolutions example
   14/3/2005 Steven Aitken - NDS8 Ltd
   http://www.nds8.co.uk
*/

function FindProxyForURL(url, host)
{
	ret = URLhash(url);
	if ( (ret % 2) == 0 ) {
		return "PROXY 10.10.10.10:8080";
	} else  {
		return "PROXY 10.10.10.11:8080";
	}
}


function URLhash(name)
{
var  cnt=0;
	var str=name.toLowerCase(name);
	if ( str.length ==0) {
	 	return cnt;	
	}
	for(var i=0;i < str.length ; i++) {
	   var ch= atoi(str.substring(i,i + 1));
		cnt = cnt + ch;
	}

	return cnt ;
}

/* 
   URLhash2( ) for directory name hash computing version.
   written by SHARP Corp in Feb 1997
   
   Objects in a same directory will be accessed via the same proxy.
   Use URLhash2( ) instead of URLhash( ) if you prefer to use persistent
   connection in HTTP 1.1

   http://www.sharp.co.jp/sample/test/img/mebius.png
   http://www.sharp.co.jp/sample/test/img/zaurus.png
   http://www.sharp.co.jp/sample/test/img/wiz.png
   <------------------------------------->
         directory name hashing here
   
*/

function URLhash2(name)
{
var  cnt=0; 
var  dirptr=0;

	var str=name.toLowerCase(name);
	if ( str.length ==0) {
	 	return cnt;	
	}

/* skip filename in directory */
	for(var i=str.length - 1;i >=0 ; i--) {
		if ( str.substring(i,i +1) == '/' ) {
			dirptr = i+1 ;
			break;
		}
	}

	for(var i=0;i < dirptr; i++) {
	   var ch= atoi(str.substring(i,i + 1));
		cnt = cnt + ch;
	}

	return cnt ;
}

function atoi(charstring)
{

	if ( charstring == "a" ) return 0x61; if ( charstring == "b" ) return 0x62;
	if ( charstring == "c" ) return 0x63; if ( charstring == "d" ) return 0x64;
	if ( charstring == "e" ) return 0x65; if ( charstring == "f" ) return 0x66;
        if ( charstring == "g" ) return 0x67; if ( charstring == "h" ) return 0x68;
	if ( charstring == "i" ) return 0x69; if ( charstring == "j" ) return 0x6a;
	if ( charstring == "k" ) return 0x6b; if ( charstring == "l" ) return 0x6c;
	if ( charstring == "m" ) return 0x6d; if ( charstring == "n" ) return 0x6e;
	if ( charstring == "o" ) return 0x6f; if ( charstring == "p" ) return 0x70;
	if ( charstring == "q" ) return 0x71; if ( charstring == "r" ) return 0x72;
	if ( charstring == "s" ) return 0x73; if ( charstring == "t" ) return 0x74;
	if ( charstring == "u" ) return 0x75; if ( charstring == "v" ) return 0x76;
	if ( charstring == "w" ) return 0x77; if ( charstring == "x" ) return 0x78;
	if ( charstring == "y" ) return 0x79; if ( charstring == "z" ) return 0x7a;
	if ( charstring == "0" ) return 0x30; if ( charstring == "1" ) return 0x31;
	if ( charstring == "2" ) return 0x32; if ( charstring == "3" ) return 0x33;
	if ( charstring == "4" ) return 0x34; if ( charstring == "5" ) return 0x35;
	if ( charstring == "6" ) return 0x36; if ( charstring == "7" ) return 0x37;
	if ( charstring == "8" ) return 0x38; if ( charstring == "9" ) return 0x39;
	if ( charstring == "." ) return 0x2e;
	return 0x20;
}

Save this file into the SYS:\ETC\PROXY\DATA directory on both BorderManager servers. The BorderManager miniweb server will then make the proxy.pac file available to your users on port 1959.

Configuring Internet Explorer

1. From Internet explorer, select Tools Menu -> Internet Options -> Connections Tab -> LAN Settings button.

2. Check the box for "Use Automatic configuration script" and set the value to http://10.10.10.10:1959/data/proxy.pac

Figure 11: Using the automatic configuration script

3. Click OK.

You can now see client requests being filled by both proxy servers, and you can turn one node off with very minimal disruption to your Internet users.

Client Configuration with DNS round-robin Load Balancing

You can split the load on both BorderManager servers by setting up client machines to resolve a proxy address of proxy.domain.com. This DNS address will have two IP addresses associated with it, and the server will return the 10.10.10.10 and 10.10.10.11 addresses alternately.

This method is suitable if you use NBM proxy services in any of the following ways:

  • CLNTRUST.EXE authenticates workstations to NBM.
  • An open system is running without access rules enabled.
  • SSL-based authentication is used.
  • Web applications require IP-address-based security.

This method is not suitable if you require very high cache performance; cached URLs will be duplicated on each server.

Example 1 - Load Balanced DNS round-robin Cluster

Figure 12: load-balanced DNS round-robin cluster

To set up a load-balanced DNS round-robin cluster,

1. Create a DNS record for proxy.domain.com.

2. Go to the Novell DHS / DHCP management console to create a new resource record.

Figure 13: DHS / DHCP console

3. Select the domain.com zone and click to create a new resource record.

Figure 14: Creating a new resource record

4. Enter the hostname as the proxy and an IP address of 10.10.10.10.

Figure 15: Entering proxy and IP information

5. Select the newly created record for proxy.domain.com and create a new resource record again.

Figure 16: New resource record for proxy.domain.com

6. This time, enter the IP address of 10.10.10.11.

7. From Internet explorer, click Tools Menu -> Internet Options -> Connections Tab -> LAN Settings.

Figure 17: Selecting LAN settings

Client Testing

As each client accesses the Internet for the first time, the DNS name proxy.domain.com will be resolved, and it will receive either the 10.10.10.10 or the 10.10.10.11 address. This DNS will then be cached to IP mapping and its requests will be routed to the same server, theoretically until it is rebooted.

Conclusion

You should now have a fully functioning BorderManager cluster with the client load split evenly between both nodes. Should one node fail, the second active node will continue to process requests with minimal disruption to users.

Note: The above solution has been derived from a laboratory based setup and should therefore be tested thoroughly before these principles are applied to a production system. While the above example has been implemented with 2 nodes, it should scale to the 32 node / 32 ISP limit imposed by the Novell Clustering software.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell