In this scenario, you have an OES 2 Linux cluster configured on physical machines. OES 2 Linux and Xen are installed and configured on each node along with Novell Cluster Services. This part of the Novell Cluster Services configuration does not differ from that of an OES 2 Linux cluster without virtualization.
You then create either Linux or NetWare virtual machines on each cluster node and configure those virtual machines to be cluster resources. You can then fail over or migrate virtual machine cluster resources (entire virtual machines) to different physical nodes in your cluster.
Figure 14-1 depicts how this setup might look. Novell Cluster Services (NCS) is installed and running on the virtual machine (VM) host server.
Figure 14-1 Virtual Machines as Cluster Resources
The following sections describe how to create a cluster resource and its cluster scripts for each virtual machine:
Novell Cluster Services includes two Xen (virtual machine) resource templates, which greatly simplify the process for creating a virtual machine cluster resource. Much of the virtual machine cluster resource configuration is performed automatically by the Xen resource templates. The two templates are named Xen_Template and XenLive_Template. Both templates perform similar functions to automatically configure the cluster resource for the virtual machine.
The XenLive template provides an additional function to allow a manual virtual machine resource migration without the need to boot or bring up the virtual machine on the cluster node where the virtual machine has been migrated. This lets clients continue to access a virtual machine that has been migrated without reconnecting or waiting for the virtual machine to boot or load on the target node.
IMPORTANT:The live migrate function is only useful for a manual virtual machine resource migration, and does not work for a virtual machine resource failover or failback.
Ensure that your Xen setup is working properly before you attempt to set up the Novell Cluster Services clustering for your virtual machines in the Xen host environment. Refer to the Virtualization with Xen to find out how to set up XEN and XEN virtual machines.
To configure a virtual machine as a cluster resource:
Open your Internet browser and enter the URL for iManager.
The URL is http://server_ip_address/nps/imanager.html. Replace server_ip_address with the IP address or DNS name of a server in the cluster or with the IP address for Apache-based services.
Specify your user name and password, specify the tree where you are installing the cluster, then click
.In iManager, select
, then click .Under
, iManager displays four links that you can use to configure and manage your cluster.Browse and select the cluster name.
On the Cluster Options page, click
.Click the
radio button to specify as the resource type you want to create, then click .Specify a name for the virtual machine resource.
In the
field, browse to the Cluster object container, then select the desired Xen template name from the list of templates in the container.The Xen templates are named
and .Select the Configuring Virtual Machine Load, Unload, and Monitor Scripts.
check box, click , then continue withThe Xen resource templates configure the virtual machine resource by automatically creating load, unload, and monitor scripts, setting failover and failback modes, and assigning the virtual machine as a resource to all nodes in the cluster.
The load, unload, and monitor scripts for virtual machine cluster resources do not need to be modified if all the following are true:
The resource name is the same as the virtual machine name.
The configuration file name is the same as the virtual machine name.
The mount point directory name is the same as the virtual machine name.
You are using the Reiser file system.
If you are not modifying the scripts, continue the setup by configuring the resource policies and the resource server assignments. For information, see Section 10.7, Configuring the Start, Failover, and Failback Modes for Cluster Resources and Section 10.8, Configuring Preferred Nodes for a Resource.
If you are modifying the scripts, continue with the following sections:
The virtual machine resource load script page should already be displayed. The load script contains commands to start the virtual machine. You can customize some commands for your specific configuration.
View and, if necessary, edit the lines in the script for your specific directory structure, mount point, configuration file, and file system type (in the Xen_Template).
See the following examples of the default Xen_Template and XenLive_Template load scripts:
The Xen_Template load script appears similar to the following example:
#!/bin/bash . /opt/novell/ncs/lib/ncsfuncs # filesystem settings export OCF_RESKEY_device=/dev/evms/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_directory=/mnt/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_fstype=reiserfs #export OCF_RESKEY_options= # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE # mount the file system exit_on_error ocf_start Filesystem # start the service exit_on_error ocf_start Xen # return status exit 0
The XenLive_Template load script appears similar to the following example:
#!/bin/bash . /opt/novell/ncs/lib/ncsfuncs # filesystem settings export OCF_RESKEY_directory=/mnt/$OCF_RESOURCE_INSTANCE # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE # start the service if [ -n "$NCS_TOFROM" ] then exit_on_error ocf_migrate_from Xen else exit_on_error ocf_start Xen fi # return status exit 0
The virtual machine resource unload script page should now be displayed. The unload script contains commands to stop the virtual machine. You can customize some commands for your specific configuration.
View and, if necessary, edit the lines in the script for your specific directory structure, mount point, configuration files, and file system type (in the Xen_Template).
Use the same values that you specified in the load script.
See the following examples of the default Xen_Template and XenLive_Template unload scripts:
Click
, then continue the setup by configuring the resource policies and the resource server assignments.For information, see Section 10.7, Configuring the Start, Failover, and Failback Modes for Cluster Resources and Section 10.8, Configuring Preferred Nodes for a Resource.
If you want to enable monitoring for the resource, continue with Configuring the Monitor Script.
The Xen_Template unload script appears similar to the following example:
#!/bin/bash ./opt/novell/ncs/lib/ncsfuncs # filesystem settings export OCF_RESKEY_device=/dev/evms/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_directory=/mnt/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_fstype=reiserfs #export OCF_RESKEY_options= # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE # stop the service ignore_error ocf_stop Xen # umount the file system ignore_error ocf_stop Filesystem # return status exit 0
The XenLive_Template unload script appears similar to the following example:
#!/bin/bash ./opt/novell/ncs/lib/ncsfuncs # filesystem settings export OCF_RESKEY_directory=/mnt/$OCF_RESOURCE_INSTANCE # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_CRM_meta_migrate_target=$NCS_TOFROM RC=0 # stop the service if [ -n $NCS_TOFROM ] then RC = `ocf_migrate_to Xen` if [ $RC -ne 0 ] then ignore_error ocf_stop Xen fi else ignore_error ocf_stop Xen fi # return status exit $RC
The Xen_Template and XenLive Template each include a resource monitoring script that you can customize. You use the script to monitor the health of a virtual machine cluster resource.
Resource monitoring is disabled by default. If you want to enable resource monitoring for a virtual machine cluster resource, you must enable it prior to customizing the resource monitoring script.
To enable resource monitoring for a virtual machine cluster resource:
In iManager, click
, then click .Browse and select the Cluster object.
Select the check box next to the virtual machine resource, then click the
link.Click the
tab, then select the check box to enable resource monitoring for the resource.Resource monitoring is disabled by default.
For the polling interval, specify how often you want the resource monitoring script for this resource to run.
You can choose to specify the number in minutes or seconds.
Specify the number of failures (
) for the specified amount of time ( ).If the resource monitor detects that the resource fails the number of times specified in the amount of time specified, a failure action initiates.
Specify whether you want the resource to be set to a comatose state, to migrate to another server, or to reboot the hosting node (without synchronizing or unmounting the disks) if a failure action initiates. The reboot option is normally used only for a mission-critical cluster resource that must remain available.
If the failure action initiates and you chose the option to migrate the resource to another server, the resource migrates to the next server in its
list. The resource remains on the server it has migrated to unless you migrate it to another server or the failure action initiates again, in which case it again migrates to the next server in its list.If the failure action initiates and you chose the option to reboot the hosting node without synchronizing or unmounting the disks, each of the resources on the hosting node will fail over to the next server in its
list because of the reboot. This is a hard reboot, not a graceful one.With resource monitoring, the
, , and modes have no effect on where the resource migrates. This means that a resource that has been migrated by the resource monitoring failure action does not migrate back to the node it migrated from unless you manually migrate it back.To view or customize the monitor script for the virtual machine’s cluster resource:
In iManager, click
, then click .Browse and select the Cluster object.
Select the check box next to the virtual machine resource that you created, then click the
link.Click the
tab, then click the link.View or edit the commands in the script that monitor the resource on the server.
You can use the same commands that would be used at the Linux terminal console.
See the following examples of the default Xen_Template and XenLive_Template monitor scripts:
Specify the
value, then click to save the script.The timeout value determines how much time the script is given to complete. If the script does not complete within the specified time, the failure action initiates based on your settings in Step 7 of Enabling Resource Monitoring. Cluster Services marks the monitor process as failed right after the defined timeout expires, but it must wait for the process to conclude before it can start other resource operations.
The Xen_Template monitor script appears similar to the following example:
#!/bin/bash ./opt/novell/ncs/lib/ncsfuncs # filesystem settings export OCF_RESKEY_device=/dev/evms/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_directory=/mnt/$OCF_RESOURCE_INSTANCE export OCF_RESKEY_fstype=reiserfs #export OCF_RESKEY_options= # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE # status of the file system exit_on_error ocf_status Filesystem # status of the service exit_on_error ocf_status Xen # return status exit 0
The XenLive_Template monitor script appears similar to the following example:
#!/bin/bash . /opt/novell/ncs/lib/ncsfuncs # service settings export OCF_RESKEY_xmfile=$OCF_RESKEY_directory/$OCF_RESOURCE_INSTANCE # status of the service exit_on_error ocf_status Xen # return status exit 0
Live migrations use the XenLive template. You can manually copy the virtual machine configuration files to the same path on each node of the cluster, or you can set up an OCFS2 file system for the configuration files. Do one of the following:
Manually copy the configuration file for the virtual machine to the same directory (the path must be the same) on each cluster node where the virtual machine will run.
Configure the OCFS2 file system on a shared disk system and copy the virtual machine configuration file to a directory on the file system. You also must ensure that all cluster nodes where the virtual machine will run have access to the OCFS2 file system on the shared disk system.
Ensure that your OCFS2 file system is working properly before you attempt to use it with Novell Cluster Services.
An overview of OCFS2 is available in Oracle Cluster File System 2
in the SUSE Linux Enterprise Server 10 SP4 Administration Guide. For detailed information about using OCFS2, see the OCFS2 Project on the Oracle Web site.
For information about setting up live migration, see Configuring a Xen VM for Live Migration with a Cluster in Novell Cool Solutions.