Novell Home

AppNote: Setting up a Business Continuity Cluster (BCC) using VMWare

Novell Cool Solutions: AppNote
By Bhogilal Hirani

Digg This - Slashdot This

Posted: 15 Jun 2005
 

Note: Bhogilal Hirani is a Technical Training Specialist from Novell UK.

Table of Contents

Setup and Installation of NetWare

Products to be installed

Install NetWare 6.5 SP2 on all the servers in the locations as per the above tree structure using the information provided in the table below.

Server name IP Address Installed products
ISCSILON 192.168.1.10 iSCSI Target, Apache2, DHCP iManager, Open SSH
LON1 192.168.1.11 Apache2, DHCP iManager, Open SSH
LON2 192.168.1.12 Apache2, DHCP iManager, Open SSH
ISCSIDUS 192.168.1.20 iSCSI Target, Apache2, DHCP iManager, Open SSH
DUS1 192.168.1.21 Apache2, DHCP iManager, Open SSH
DUS2 192.168.1.22 Apache2, DHCP iManager, Open SSH

Post Installation checklist

ISCSILON LON1 LON2 ISCSIDUS DUS1 DUS2
AUTOEXEC.NCF
REM AFP, NFS, CIFS, STARTX
SERVER -NL
CONNECTIVITY (NRM & iMANAGER)
eDir Replica MASTER ROOT R/W ROOT R/W ROOT R/W ROOT R/W ROOT R/W ROOT
Time Source SINGLE SECONDARY (10.1.1.10) SECONDARY (10.1.1.10) SECONDARY (10.1.1.10) SECONDARY (10.1.1.10) SECONDARY (10.1.1.10)
VMTOOLS
Access NRM
Access iManager
Timesync
Replica sync

TIP:

Ensure that iManager can be run from each server.

If you can't login to iManager, try authenticating using the CN=admin,O=novell and then configure iManager as follows:

  1. Click on Configure.
  2. Go to iManager configuration | Portal | Properties.
  3. Change the Portal Containers field to the appropriate container (o=novell).
  4. Save the changes.
  5. Repeat for iManager on other servers.

Update the host file

Add the following entries in each sys:etc/hosts file

192.168.1.10 	iscsilon.training.novell.com 	iscsilon 	bcc-tree
192.168.1.11	lon1.training.novell.com 		lon1
192.168.1.12 	lon2.training.novell.com 		lon2
192.168.1.20 	iscsidus.training.novell.com 	iscsidus 	bcc-tree
192.168.1.21 	dus1.training.novell.com 		dus1
192.168.1.22 	dus2.training.novell.com 		dus2

#The following IP addresses will need to be assigned to the master IP address for each cluster

192.168.1.15						cllon
192.168.1.25						cldus

Create iSCSI Targets

  1. On the target servers, load NSSMU and create the following iSCSI partitions providing the appropriate label for easy identification.
  2. For each iSCSI partition, make the appropriate server(s) trustees of the target object.
Server Type Size Label Trustee
ISCSILON iSCSI 20 MB SBD LON LON1
iSCSI 400 MB DATA LON LON1, LON2 , DUS1, DUS2
iSCSI 510 MB Mirror DATA DUS LON1, LON2 , DUS1, DUS2
ISCSIDUS iSCSI 20 MB SBD DUS DUS1
iSCSI 500 MB DATA DUS LON1, LON2 , DUS1, DUS2
iSCSI 410 MB mirror DATA LON1 LON1, LON2 , DUS1, DUS2

TIPS:

  1. Use ISCSITAR TARGETS console commands to view the iscsi partitions. If you do not see the targets displayed check the following:
    • Error messages on the logger screen when executing TON.NCF ( I saw public symbol errors relating to ldap).
    • Load TBX on the target server and delete the iscsi secret store file(sys:etc/iscsi.lss).
    • Execute TOFF.NCF and then TON.NCF
    • On the logger screen you will see the msg:
      LDAP credentials have not been initialized on the server .......
    • You are not required to do this LDAP setup; it is only necessary if you want security with respect who is allowed to access a target.
    • Launch NRM for the target server and authenticate (.admin.novell OR cn=admin,o=novell).
    • Click on iSCSI Services.
    • Click on LDAP.
    • Verify the Target server's context and enter the password for the Admin.
    • Execute TOFF.NCF and then TON.NCF
  2. Create an NCF (INITSTRT.NCF) file to connect to the targets and copy the file to each node i.e LON1, LON2, DUS1 and DUS2
    • ION.NCF
    • ISCSINIT CONNECT 192.168.1.10
    • (I would add a delay here)
    • ISCSINIT CONNECT 192.168.1.20
    • ISCSINIT INFO
  3. Start NSSMU on one node per cluster, select the devices on the iSCSI targets, flag them "sharable for clustering" (F6) and do a device init (F3).
  4. Do not establish mirroring at this stage.
    The main reason for this is that if mirroring is established you cannot created pools on the mirror set. It is best to create the pools first (done towards the end) and then establish mirroring.

Installing Novell Cluster Services

Execute the previously created INITSTRT.NCF file on LON1, LON2, DUS1 and DUS2 and create the clusters CLLON and CLDUS using the information in the table below.

Cluster name Location nodes master ip address resource
CLLON .clu.res.lon.novell LON1 & LON2 192.168.1.15
CLDUS .clu.res.dus.novell DUS1 & DUS2 192.168.1.25

TIPS:

  1. On my laptop, NWDEPLOY would fail to launch. Renaming the JRE & NI directory in the C:\Program Files\Common Files\Novell directory cured the problem.
  2. Map to LON1 and DUS1 before starting the install.

Install the Clusters for the LONDON and the DUESSELDORF site.

  1. Run NWDEPLOY from the OS CD.
  2. Click on Install/Upgrade Clusters and then on Install or Upgrade a Cluster. Click Next and Next again.
  3. On the "NCS Cluster Selection" screen supply the following information and click Next.

  4. On the "NCS Cluster Node Modification" screen browse to the LON1 and LON2 servers. Click Add.

  5. Click Next.
  6. Supply 192.168.1.15 as the ip address for the Master IP Address Resource. Click Next.

  7. Click Next.
  8. On the "NCS Shared Media Selection" screen, ensure that the "Does the cluster have shared media? = yes" is automatically detected. If not, check your iSCSI connections making sure that the correct device is selected in the "Select the storage device with shared media". This should be the 20 Mb iSCSI partition with the label of SBD LON.

  9. Click Next and Next again.
  10. When successfully installed, click Close.
  11. Ensure that cluster services loads on the LON1 and LON2 nodes of the CLLON cluster.
  12. For each cluster node, edit the AUTOEXEC.NCF and modify the following entry:
    REM LDNCS.NCF

Repeat the above steps to install the cluster CLDUS supplying the information in the table that is specific to it.

NOTE: If the setup is using VMWare on a Linux host, then you will notice that the IP address of the Master IP Address resource (192.168.1.15 and 192.168.1.25) will give an error with "IP address already in use" message on the logger screen when cluster services loads. This error is due to the setup being on VMWare. A gratitious arp is being made when the IP address is bound and the LAN card eth0 bounces the IP Address back to the server

To work around this problem, the load script for the Master IP Address Resource needs to be modified and a DELAY 10 inserted in the load script between the "Add secondary IP address 192.168.1.15" (and 192.168.1.25 for the CLDUS cluster) and "SET allow ip address duplicates =off".

As the script is not modifiable, do the following to add the DELAY=10 statement:
Unload cluster services

  1. On LON1, copy the NCS directory in the sys:public/mgmnt/consoleone/1.2/resources and snapin directory to another location.
    You can also use the ConsoleOne Snapin Selector by Wolfgang Schreiber; very handy.
  2. Start ConsoleOne on LON1.
  3. Browse to the cluster object.
  4. Go to the properties of the master IP address resource.
  5. Click on the "other" tab.
  6. Check the value for the NCS:CRM Failover mode attribute.
  7. Copy back the NCS directory into the resources and snapin directory, restart ConsoleOne on LON1.
    1. Value should be 2147483697 decimal = 800000031 hex
    2. Take off the MSB to leave 31 Hex
    3. Convert 31Hex to decimal = 49
    4. Change value of attribute to 49
    5. Click Apply and OK
  8. Copy back the NCS directory into the resources and snapin directory restart consoleOne on LON1.
  9. Add DELAY 10 after the "Add secondary ip address <IP Address>.
  10. Click Apply and then OK.
  11. Load Cluster Services on LON1.
  12. Check that the IP address is bound without any errors (display secondary ipaddress).
  13. Change the value of the NCS:CRM Failover mode attribute to 2147483697

Repeat the steps on DUS1 for the CLDUS cluster

Install DirXML

On LON1 and DUS1 install the DirXML Engine and eDirectory driver from the DirXML Starter Pack version 1.1a

  1. Mount the CD on server LON1.
  2. Run STARTX on the server console.
  3. Start | Install | Add | <vol>:\XMLSTRPK\NW\products.ni
  4. Click Next and Next again.
  5. Click on I Accept to accept the license.
  6. On the "Components Screen" select the "DirXML Engine & Drivers" & "Novell iManager plugins for DirXML". Click Next.
  7. On the "Schema Extension" screen, authenticate using cn=admin,o=novell. Click Next.
  8. On the "Components Screen" select "DirXML Driver 1.1.2 for eDir". Click Next.
  9. Click Finish.
  10. If prompted, DO NOT overwrite newer files.
  11. Click Close when the install is complete.
  12. If prompted, restart Web Services.

TC4STOP
(Toggle to the logger screen and wait until "....bootstrap exited successfully")
TOMCAT4
(Toggle to the logger screen and wait until "jk running....")

TIPS:

  • Switch to the logger screen and wait for the message "jk running.."
  • Don't forget to repeat the install for DUS1.

Install Novell Business Continuity Cluster Services (BCC)

  1. Run INSTALL.EXE.
  2. Click Next.
  3. Click on I Accept to accept the License.
  4. On the "Components Screen" select "Novell BCC" & "iManager snapins for Novell NetWare based management servers". Click Next.
  5. Click Next on the "Upgrade Reminder" screen. (The modules were upgraded during the OS install.)
  6. On the "Select Cluster" screen browse to and select the CLLON cluster object.

  7. Click Next.
  8. On the "Select Nodes" screen, select the LON1 and the LON2 nodes.


  9. Click Next.
  10. On the "Select Server" screen browse to the LON1 as the server to install the iManager snapins to.
  1. Click Next and then Finish.
  2. Click Close upon completion of the installation.
  3. Edit the AUTOEXEC.NCF file on each node and REM the LDBCC.NCF.

Repeat the BCC installation steps for the CLDUS cluster and install the iManager snapins to DUS1.

Configuring the eDirectory Drivers

Before you begin the configuration of the eDirectory drivers ensure that the following Organizational Units do exist:

OU= FROMDUS the RES.LON.NOVELL
OU= FROMLON in the RES.DUS.NOVELL

They will serve as the landing zone for the eDirectory objects created by the DirXML drivers.

  1. Start iManager for the DirXML node in the CLLON cluster (LON1).
    https://192.168.1.11/nps/iManager
  2. Under DirXML Management click Create Driver.
  3. Select In a new Driver Set and provide the following information:

    Name: BCC lon-dus driverset
    location: . clu.res.lon.novell
    Server: LON1

    Uncheck the create a new partition on this Driver set.


  4. Click Next.
  5. Click OK on the warning prompt.
  6. In the Import a pre-configured driver from the server (.XML file), select the BCCClusterResoruceSyncronisation.XML. Click Next.

  7. On the "Create Driver" screen enter the following information (screenshots follow):
    • Driver Name : lon2dus driver
    • Name of the SSL Certificate: none
    • Name of other DirXML Node: 192.168.1.21
    • Port Number for this driver: 2002
    • Full Distinguished Name (DN) of this cluster: CLLON.CLU.RES.LON.NOVELL
    • Full Distinguished Name (DN) of other cluster:CLDUS.CLU.RES.DUS.NOVELL
    • Context where cluster-enabled pool and volume objects will be synchronized for this cluster: fromdus.res.lon.novell
      (This is the "Landing zone" for the objects to be synchronized from the other cluster)
    • Parent container context of cluster-enabled pool and volume objects for other cluster: clu.res.dus.novell
      (This is the context of where the cluster enabled pool and volume objects reside in the other cluster)
    • Name of other cluster's tree: BCC-TREE


  8. Click Define Security Equivalence.
  9. Click Add, browse to the Admin.novell user and then click OK.
  10. Click Exclude Administrative roles.
  11. Click Add.
  12. Browse to the Admin.novell user and then click OK.
  13. Click Next.
  14. On the Summary screen click Finish.

Repeat above steps and create a driver set and driver for the other cluster (CLDUS) supplying the relevant information.

  • DriverSet name: BCC dus2lon Driverset
  • Driver Name : dus2lon driver
  • Name of the SSL Certificate: none
  • Name of other DirXML Node: 192.168.1.11
  • Port Number for this driver: 2002
  • Full Distinguished Name (DN) of this cluster: CLDUS.CLU.RES.LON.NOVELL
  • Full Distinguished Name (DN) of other cluster: CLLON.CLU.RES.DUS..NOVELL
  • Context where cluster-enabled pool and volume objects will be synchronized for this cluster:fromlon.res.dus.novell
    (This is the "Landing zone" for the objects to be synchronized from the other cluster)
  • Parent container context of cluster-enabled pool and volume objects for other cluster: clu.res.lon.novell
    (This is the context of where the cluster enabled pool and volume objects reside in the other cluster)
  • Name of other cluster's tree: BCC-TREE

Start the DirXML drivers

  1. Load iManager for server LON1.
  2. Under Roles and Tasks, click on DirXML Management.
  3. Click on Overview.
  4. Click Search to "Search Entire Tree."
  5. Click on the icon and select the Start Driver option.
  6. Repeat the steps for both driver for DUS1.

Configuring BCC

On each node ensure that Cluster Services is running and then load BCC by executing LDBCC.NCF on each node.

BCC Configuration

  • Enable Clusters for BCC.
  • Add resource scripts "search and replace values."
  • Add Cluster peer credentials.
Enable Clusters for BCC

  1. Ensure the DirXML drivers are running.
  2. Start iManager for DirXML node (LON1).
  3. Click on Cluster Administration | Configuration.
  4. Browse to the CLLON cluster object.
  5. Click on Properties.
  6. Click on the Business Continuity Tab.
  7. Check mark the "Enable Business Continuity Features."
  8. Continue with "Adding Resource Script Search and Replace" Values.

Adding Resource Script Search and Replace values

Note

The search and replace values can either be cluster specific (apply to all resources in the cluster) or resource specific. Complete the following steps for cluster specific configuration. Typically you will have both clusters in different subnets and each cluster will have multiple resources, in which case it is easier to configure the search and replace values to be cluster specific.

For the purpose of this demo, I decided to configure the values for each individual resource when the resource is BCC enabled (see the steps described in "Configuring cluster resources for Business Continuity").

Repeat the steps described in the "BCC Configuration" to BCC Enable the CLDUS cluster and create Resource Script Search and Replace values.

If everything is configured correctly, you should see "All Peer Connections are up" in the cluster management. If not, the connection credentials will need to be setup or verified. Refer to the steps described in "Setting up connection credentials."


Setting up connection credentials

For Cluster CLLON, set the credential of Admin.novell to authenticate to cluster CLDUS and vice versa.

  1. Cluster Administration | Configuration.
  2. Browse to the CLLON cluster object and click Properties.
  3. Click on the Business Continuity tab.
  4. Under Connections, Check the CLDUS cluster. Click Edit.
  5. Supply Admin.novell with novell as the password. Click OK.

Creating a resource for in the CLLON cluster

  1. On LON1 load NSSMU.
  2. Select Pools.
  3. Press Insert.
  4. Enter LONP as the pool name and press Enter.
  5. Select the appropriate device to create the partition on and press Enter (use ISCSITAR TARGETS on the target server to work out the correct device/target).
  6. Select the default size presented.
  7. Leave the Virtual Server Name as the default (this should match the entry in the sys:etc/hosts file).
  8. Assign an IP address of 192.168.1.16 (this should match the entry in the sys:etc/hosts file).
  9. Press Create.
  10. Create a volume LONV in the pool LONP
  11. Repeat the above steps to create a resource for the CLDUS cluster

    Pool=DUSP
    IP=192.168.1.26
    Volume=DUSV

Establish Mirroring for the LONP_SERVER Resource

  1. Load NSSMU on LON1.
  2. Select Partitions.
  3. Select the Partition to be mirrored (Partition size should be 400 MB with a pool LONP on it).
  4. Press F3 to select the mirror options.
  5. Select the appropriate partition residing on the ISCSIDUS server.
  6. Issue the MIRROR STATUS console command to view the mirroring status and wait for it to complete. The message " Mirror object 0x.. is synchronized" denotes mirroring completion.

Configuring cluster resources for Business Continuity

The resources LONP_SERVER and DUSP_SERVER will need to be BCC enabled for them to be able to failover across clusters.

Complete the following steps to BCC enable the LONP_SERVER cluster resource.

  1. Start iManager.
  2. Click on Cluster Administration | Configuration.
  3. Browse to the CLLON cluster object.
  4. Check mark the LONP_SERVER resource and click Properties.
  5. Click on the Business Continuity tab.
  6. Check mark the Enable Business Continuity Features.
  7. In the Resource Script Replacements section, click New.
  8. In the search Value enter 192.168.1.27
  9. In the Replacement Value enter 192.168.1.16


  10. Click OK.
  11. From the Available Peer Clusters list check mark both the CLLON and CLDUS cluster.
  12. Click OK.
  13. The icon of the resource will now change to denote that it is now BCC enabled.


  14. Click on Cluster Administration | Configuration.
  15. Browse to the CLDUS cluster object.
  16. Check mark the LONP_SERVER resource and click Properties.
  17. Click on the Business Continuity tab.
  18. In the Resource Script Replacements section, click New.
  19. In the search Value enter 192.168.1.16
  20. In the Replacement Value enter 192.168.1.27
  21. Click OK.

Testing the Migration

  1. In iManager click on Cluster Administration | Management.
  2. Click on the CLLON tab.
  3. Check mark the LONP_SERVER resource and click on BCC migrate.



  4. On the subsequent screen, select CLDUS as the cluster the resource is to be migrated to. Click OK.
  5. The resource changes to a secondary state on the CLLON cluster.


  6. The resource will begin to load on the Preferred node on the CLDUS cluster i.e on node DUS1.
  7. While on the CLDUS cluster, check the resource load script for the resource.

Repeat the steps described above in the "Configuring cluster resources for Business Continuity" to BCC enable the resource DUSP server for the CLDUS cluster.

  1. Start iManager.
  2. Click on Cluster Administration | Configuration.
  3. Browse to the CLDUS cluster object.
  4. Check mark the DUSP_SERVER resource and click Properties.
  5. Click on the Business Continuity tab.
  6. Check mark the Enable Business Continuity Features.
  7. In the Resource Script Replacements section, click New.
  8. In the search Value enter 192.168.1.17
  9. In the Replacement Value enter 192.168.1.26
  10. Click OK.
  11. From the Available Peer Clusters list check mark both the CLLON and CLDUS cluster.
  12. Click OK.
  13. The icon of the resource will now change to denote that it is now BCC enabled.
  14. Click on Cluster Administration | Configuration.
  15. Browse to the CLLON cluster object.
  16. Check mark the DUSP_SERVER resource and click Properties.
  17. Click on the Business Continuity tab.
  18. In the Resource Script Replacements section, click New.
  19. In the search Value enter 192.168.1.26
  20. In the Replacement Value enter 192.168.1.17
  21. Click OK.

Testing the Migration of the LONP_SERVER Resource

  1. In iManager click on Cluster Administration | Management.
  2. Click on the CLLON tab.
  3. Check the LONP_SERVER resource and click on BCC migrate.
  4. On the subsequent screen, select CLDUS as the cluster the resource is to be migrated to and click OK.
  5. The resource changes to a secondary state on the CLLON cluster as the unload script executes.
  6. The resource should then begin to load on the Preferred node on the CLDUS cluster i.e on node DUS1.
  7. While on the CLDUS cluster, check the resource load script for the resource and confirm that the IP Address has been changed from 192.168.1.16 to 192.168.1..27
  8. Repeat the Testing the Migration steps for the DUSP_SERVER resource.

TIPS:

For stability under VMWare increase the tolerance for the cluster heartbeat to 16 seconds on each cluster For each resource, add the commands SET ALLOW IP ADDRESS DUPLICATES =ON before the ipaddress is added and then set it to OFF after the address is bound. - VMWare issue.

BCC Demo

  1. Start each server (check timesync and eDir sync).
  2. On the cluster nodes, execute CONN.NCF to connect to the appropriate targets.
  3. Load LDNCS.NCF on LON1, LON2, DUS1 and DUS2.
  4. Load BCC.NCF on LON1, LON2, DUS1 and DUS2.
  5. To speed up the demo load iManager from the following servers:
    • Load iManager on LON1 (https://192.168.1.11/nps/iManager).
    • Load iManager on DUS1 (https://192.168.1.21/nps/iManager).
    • Load iManager on ISCSILON (https://192.168.1.10/nps/iManager).
    • Load iManager on ISCSIDUS (https://192.168.1.20/nps/iManager).
  6. On each DirXML node (LON1 and DUS1) check that the DirXML driver is running.
    DirXML Management | Overview | Start driver
  7. Ensure that the resource LONP_SERVER is active on LON1.
  8. Ensure that the resource DUSP_SERVER is active on DUS1.
  9. Migrate the LONP_SERVER resource from CLLON cluster to CLDUS cluster .
    1. Cluster Administration | Management
    2. Browse to the CLLON cluster.
    3. Check mark the LONP_SERVER resource.
    4. Click on BCC Migrate and then click OK.
      The resource goes into an "Unloading" state on the CLLON cluster and "loading" on CLDUS cluster.
      (On DUS1 switch to the logger screen to see the activity)
    5. The resource should come active on server DUS1 in the CLDUS cluster.
    6. View the load script of the migrated resource on the CLDUS cluster. Note the change of the ipaddress from 192.168.1.26 to 192.168.17
    7. Show the "resource script replacement" configuration for the CLDUS cluster.
      1. Cluster Administration | Configuration
      2. Browse to the CLDUS cluster.
      3. Check mark the LONP_CLUSTER resource and click on Properties.
      4. Click on the Business Continuity Tab.
      5. Check mark the replacement script and click Edit to show the search and replacement value.

Downing the BCC environment

  1. Down each cluster (CLUSTER DOWN).
  2. Unload BCC on each BCC node (ULDBCC).
  3. Unload NCS on each node (ULDNCS).

BCC Console Commands

Issue the following commands on the BCC console:

  • CLUSTER HELP
  • CLUSTER VIEW – Peer clusters
  • CLUSTER CREDENTIALS – credentials to connect to the peer clusters
  • CLUSTER DISABLE <resource> – Disables BCC for the specified resource
  • CLUSTER ENABLE <resource> – Enable BCC for the specified resource
  • CLUSTER MIGRATE source/resource destination/node
    cluster migrate cllon/lonp_server cldus/dus1
  • CLUSTER RESETRESOURCES – All resources on this cluster will be set to offline and secondary
  • CLUSTER RESOURCES – State and location of cluster resources
  • CLUSTER STATUS – State and location of cluster resources


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell