Novell Home

Configuring a Xen VM for Live Migration within a Cluster

Novell Cool Solutions: Feature
By Jason Record

Digg This - Slashdot This

Posted: 28 Sep 2007
 

Problem

By default, migrating a Xen Virtual Machine (VM) resource causes it to shutdown on the current node and restart on the new one. Once you configure a Xen VM within the High Availability Storage Infrastructure (HASI), how do you configure the Xen VM resource to live migrate among the cluster nodes?

Solution

Before you can configure the Xen VM resource for live migration, there are a couple of prerequisites that need to be met:

  1. Make sure all nodes are installed with SUSE LINUX Enterprise Server 10 with Service Pack 1. Newer updates may work too, but were not considered for this writing.
  2. Configure an HASI cluster, as documented in the following link.
  3. http://wiki.novell.com/index.php/SUSE_Linux_Enterprise_Server#High_Availability_Storage_Infrastructure

The live migration configuration steps are summarized as follows:

  1. Configure xend to allow live VM migration on all cluster nodes
  2. Activate live migration on the Xen VM resource in the cluster
  3. Method 1: Modify an existing Xen VM resource
    Method 2: Create a new Xen VM resource
  4. Create a place constraint to migrate the Xen VM to a new target node

Solution Details

Configure xend to allow live VM migration on ALL cluster nodes

  1. Edit the /etc/xen/xend-config.sxp.
  2. Change these options

    #(xend-http-server no)
    #(xend-relocation-server no)
    #(xend-address '')
    (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
    

    to this

    (xend-http-server yes)
    (xend-relocation-server yes)
    (xend-address '')
    (xend-relocation-hosts-allow '')
    
  3. Type rcxend restart to apply the changes.
  4. Type netstat -nlp | grep 800 to confirm that the default ports 8000 and 8002 are now listening.
  5. Repeat steps 1 through 3 above on all nodes in the cluster.

Activate live migration on the Xen VM resource in the cluster

Method 1: Modify an existing Xen VM resource

  1. Find the Xen VM resource ID
    1. Use cibadmin -Q | less to view the cib database that contains the Xen VM resource <primitive> XML tags. XML tags are the bracketed keyword pairs like: <primitive></primitive>
    2. Next, look for 'class="ocf" type="Xen" provider="heartbeat"'
    3. Finally, you need to copy the entire record within the <primitive> tags.
    4. For example:
    5. cibadmin -Q | less
      
      --snip--
      <primitive id="vm1" class="ocf" type="Xen" provider="heartbeat">
        <operations>
         <op id="vm1-op-01" name="monitor" interval="10s" timeout="60s" prereq="nothing"/>
         <op id="vm1-op-02" name="start" timeout="60s" start_delay="0"/>
         <op id="vm1-op-03" name="stop" timeout="300s"/>
        </operations>
        <instance_attributes id="vm1">
         <attributes>
          <nvpair id="vm1-attr-01" name="xmfile" value="/etc/xen/vm/vm1"/>
          <nvpair id="vm1-attr-02" name="target_role" value="started"/>
         </attributes>
        </instance_attributes>
      </primitive>
      --snip--
      
  2. Create an XML file that modifies the Xen resource meta data
    1. You really only need the <primitive> tags that identify which resource you are modifying.
    2. Within the <primitive> tags, you need to add the meta data attributes with their appropriate tags.
    3. The following example creates a file called live-migrate.xml, with the needed XML tags. You can copy and paste it on a terminal command line to create the file.
    4. cat << EOF1 > live-migrate.xml
      <primitive id="vm1" class="ocf" type="Xen" provider="heartbeat">
        <meta_attributes id="vm1-meta-01">
          <attributes>
            <nvpair id="vm1-meta-attr-01" name="allow_migrate" value="true"/>
          </attributes>
        </meta_attributes>
      </primitive>
      EOF1
      
  3. Import the XML file to modify the existing resource
  4. cibadmin -M -o resources -x live-migrate.xml
    
  5. This step only needs to be done on one cluster node

Method 2: Create a new Xen VM resource

  1. Create a file called vm1-with-migrate.xml that includes all the necessary XML tags to create the Xen VM resource.
  2. The following example creates a file called vm1-with-migrate.xml, with the needed XML tags. You can copy and paste it on a terminal command line to create the file.
  3. cat << EOF2 > vm1-with-migrate.xml
    <resources>
      <primitive id="vm1" class="ocf" type="Xen" provider="heartbeat">
        <operations>
          <op id="vm1-op-01" name="monitor" interval="10s" timeout="60s" prereq="nothing"/>
          <op id="vm1-op-02" name="start" timeout="60s" start_delay="0"/>
          <op id="vm1-op-03" name="stop" timeout="300s"/>
        </operations>
        <instance_attributes id="vm1">
          <attributes>
            <nvpair id="vm1-attr-01" name="xmfile" value="/etc/xen/vm/vm1"/>
            <nvpair id="vm1-attr-02" name="target_role" value="started"/>
          </attributes>
        </instance_attributes>
        <meta_attributes id="vm1-meta-01">
          <attributes>
            <nvpair id="vm1-meta-attr-01" name="allow_migrate" value="true"/>
          </attributes>
        </meta_attributes>
      </primitive>
    </resources>
    EOF2
    
  4. Make sure there is not a Xen VM resource called "vm1" already defined in the cluster. You can check by running the following:
  5. cibadmin -Q | grep id=\"vm1\"
    
  6. Finally, import vm1-with-migrate.xml file to create the resource
  7. cibadmin -C -o resources -x vm1-with-migrate.xml
    
  8. This step only needs to be done on one cluster node
  9. There are a few things you should notice
    1. The "target_role" is set to "started". This means that as soon as you import the Xen VM resource file vm1-with-migrate.xml, the Xen VM will be powered on.
    2. The "stop" operation has a 300 second (5 minute) timeout. This tells the cluster to give the Xen VM resource up to five minutes to stop before the cluster determines the resource has failed. You can play with this value to determine what is best for your environment, but remember the VM needs sufficient time to shutdown.

Create a place constraint to migrate the Xen VM to a new target node

The place constraint tells the policy engine where the Xen VM resource should be running. As such, it initiates the live migration. There are two ways to create a place constraint: command line or the graphical user interface.

Command Line Method (crm_resource)

Run the command crm_resource -M -r vm1 -H node1, where vm1 is the Xen resource name/ID, and node1 is the node on which you want the Xen VM running. To migrate the Xen VM to a different node, simply repeat the command with a different node name.

Graphical User Interface Method (hb_gui)

Creating a new place constraint

  1. Start hb_gui and select Resources, Add New Item.
  2. Change the Item Type to "place", Clock OK.
  3. Change the ID to "vm1-migrate", the Resource to "vm1" and the Score to "INFINITY"; then click OK.

  4. Add Place Constraint


  5. Highlight the "vm1-migrate" place constraint, then click Add Expression.
  6. Set Attribute to "#uname", Operation to "eq" and Value to the migration target node. You will just select the node to which you want the Xen VM to migrate.

  7. Add Expression


  8. Click OK, then click Apply.

Modifying the place constraint to migrate the resource to a new node

  1. Highlight the "vm1-migrate" place constraint.
  2. In the Attributes box, highlight the "#uname eq node" expression.
  3. Click in the Value field with the current node, which will change to a drop down list.

  4. Change Expression


  5. Select the new node to which you want the Xen VM migrated.
  6. Click Apply.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell