Novell Home

AppNote: iFolder on Open Enterprise Server Linux Cluster using iSCSI

Novell Cool Solutions: AppNote
By Yatin P. Manerker

Digg This - Slashdot This

Posted: 28 Jun 2005
 

Yatin P. Manerker
ymanerker@novell.com
Senior Software Engineer

This AppNote describes how to configure iFolder on iSCSI-based Linux cluster. The objective is to provide an alternative to original cluster hardware which is more costly than an ISCSI. Even setting up and configuring any service is very simple on iSCSI.

Table of Contents

1.0 Introduction
2.0 Minimum system requirements
3.0 Installation
3.1 NetWare Installation
3.2 Linux Installation
4.0 iSCSI Target Configuration on NetWare 65SP3
5.0 iSCSI Initiators Configuration on Linux Server
6.0 Configuring clustering services on OES Linux
7.0 Configuring iFolder 2.X on OES Linux for clustering
8.0 Configuring iFolder files on cluster volume
9.0 Creating cluster resource using iManager

1.0 Introduction

Clustering is a technique in which two or more servers are interconnected and can access a common storage pool. Cluster Services enables you to combine a number of servers together into a single group, known as a cluster. One of the primary benefits of Novell Cluster Services is that it provides failover capabilities to your server clusters. If one server in the cluster should happen to fail, another server automatically recovers the downed-server's resources and runs in its place.

iSCSI - The iSCSI protocol combines the use of block-level data movement with TCP/IP networks. By allowing SCSI commands to travel through IP networks, high-speed IP networking technology can carry data from storage units to servers anywhere throughout a corporate network .Also referred to as IP storage.

The information given below is basically to setup and configure iSCSI for hybrid cluster. In this setup NetWare server is used as iSCSI target and Linux server is used as iSCSI initiators. On configuring the target and initiator, Linux server can access the iSCSI partition and mount it for Linux specific file system. Using Novell cluster service iSCSI partition can be accessed on multiple Linux servers.

2.0 Minimum system requirements

Three server class machines:

  • Two machines for Linux
  • One for NetWare

Software requirements:

  • SLES9 OES
  • NetWare server 6.5 support pack 3

3.0 Installation

Important:

Ensure that all three machines are in the same tree otherwise clusters will not work.

  • Make NetWare server as a Master server.

  • Add both the Linux server to NetWare.

  • On configuring edirectory for Linux server, point to remote LDAP for authentication (remote LDAP will be master LDAP server i.e. NetWare in the same tree).

  • During OES Linux installation do not configure iFolder2.x and Novell cluster services (NCS). This needs to be configured on completion of OES installation.

  • Use EVMS to create partition on Linux machine.

3.1 NetWare Installation

  • During the installation process of NetWare server 6.5 SP3, "iSCSI Target" checkbox must be checked.

3.2 Linux Installation

  • During the installation process of Linux server, "Various Linux Tools" checkbox must be checked. (This will select iSCSI on Linux server).

  • List of OES products required: eDirectory, iFolder2.x, iManager, Novell cluster services.

4.0 iSCSI Target configuration on NetWare 6.5 SP3

  • Type "NSSMU" at the NetWare server console.

  • Select partition from the Main menu.

  • Insert to create new partition and select iSCSI and create partition of 5GB.

  • After successful creation of iSCSI partition exit NSSMU.
  • At the NetWare server console type "ton" to create iSCSI object in edirectory.

  • Open NetWare ConsoleOne and check for iSCSI object. iSCSI object will look like
    iscsi-80ce1060-d68e-d911-bb75-0080481cd2d5.
  • In the ConsoleOne right click the iSCSI object and select properties.

  • Click Add Trustee and select the Server objects (Includes both target server and Initiators).

  • Add all three servers as trustee to iSCSI object and assign all rights.


  • Initialize iSCSI target by typing the following commands on NetWare server console:
    Editor's Note: This step is not absolutely necessary. This initialization is done automatically, by installing the iSCSI SAN server (NetWare6.5 pattern deployment) - so this would already have been done because of earlier steps to install the iSCSI SAN server.

    tinit cn=<admin user name>,<type full admin user context> <admin passwd> cn=<iSCSI target server name>,<type full iSCSI target server context>

    Example: tinit cn=admin,o=novell novell cn=myNetWareserver,o=novell

  • Run TON at iSCSI target server console.

  • Run ION at iSCSI target server console.

  • Run iscsinit connect <IP address of the iSCSI target server> at the iSCSI target server console

  • Type list devices on NetWare server console.


  • Run iscsitar targets at NetWare server console to view iSCSI target as given below.


    Where iqn.1984-08.com.novell:iscsi-80ce1060-d68e-d911-bb75-0080481cd2d5.novell is the iSCSI target name.

  • Now iSCSI target is initialized on NetWare server.

5.0 iSCSI initiator configuration on Linux server

  • Make sure that iSCSI is installed on both the Linux Machines.

  • Follow these steps on both the machines
Step 1:
Open iscsi.conf file on Linux server and enter Discovery address and Target name. Discovery address is the IP address of NetWare server (iSCSI target) and target name is the name of iSCSI as shown above.
  • vi /etc/iscsi.conf

  • b. At the end of file type

    Discovery address= <NetWare server IP address>

    TargetName= <iscsi target name>

    Example:

    DiscoveryAddress =10.1.1.20

    Target Name= iqn.1984-08.com.novell:iscsi-80ce1060-d68e-d911-bb75-0080481cd2d5.novell

  • To get target name on Linux open NRM of NetWare server (https://<IP address of NetWare>:8009) on Linux initiator.

  • Click on console screens -> system console.

  • To display target name type on NetWare console iscsitar targets.

  • Copy target name and paste in iscsi.conf file.

Save and close this file.

Step 2:
Open iSCSI initiator file and enter DNS name of target server (target is the NetWare server). Remove the fields after colon (:) and enter DNS name of server followed by context.
  • vi /etc/initiatorname.iscsi

    Example: initiatorName=iqn.1987-05.com.cisco:cn=mynetwareserver,o=novell

Save and close this file.

Step 3:
Start ISCSI on Linux servers.
  • On Linux terminal type /etc/init.d/iscsi start

  • Popup will come up showing iSCSI partition.
  • Select iSCSI partition and create ext3 file system and give about 4GB size. Do not configure mount point. Leave mount point as blank. (If you configure mount point, entry will be made in fstab and partition will be mounted automatically after system reboots).
Step 4:
Create directory /mnt/ifolder on Linux machines.
    (This directory will be used for mounting iSCSI partition)

Important: iSCSI should start automatically once server reboots. This is very important because in case of cluster failover iSCSI should start automatically after rebooting.

Step 5:
To start iSCSI automatically enter on the terminal of both the Linux machines:
    chkconfig iscsi on

6.0 Configuring Cluster on OES Linux server through YaST

  • Type yast2 ncs on Linux terminal.
Step 1:
NCS LDAP Server Configuration screen is displayed.
  • In Local or Remote Directory Server, select the Remote System checkbox.

  • Directory server address : Input IP address of the NetWare server (Master LDAP server).

  • Admin name with context: Enter the admin name with context.

  • Admin password: Enter the admin password.

  • Click Next.
Step 2:
NCS configuration screen is displayed.
  • In New or Existing cluster, select New Cluster checkbox.

  • Cluster FDN: Input cn=clusternew.o=novell

  • Cluster IP address: Enter separate IP address for new cluster.(with valid DNS name)
Click Next.
Step 3:
NCS Configuration screen is displayed.
  • Displays the Name and the IP address of the node.

  • Select the checkbox of Start Clustering Services now.
Click Next
Step 4:
The Saving NCS Configuration screen saves the cluster settings.
  • Click Next

  • Go to second Linux server and configure cluster as shown in step 1 and select existing cluster and give same name for cluster as given earlier.
  • Click next to continue.

  • This will configure Novell cluster services on Linux.

7.0 Configuring iFolder 2.x on Linux

Before configuring iFolder2.x you should mount iSCSI partition on Linux machine. You should create a directory /mnt/ifolder on both the Linux machine.

Step 1:
To mount iSCSI partition on Linux machine give command in terminal.
  • mount /dev/evms/sdx1 /mnt/ifolder Where 'x' is a or b or c etc.

    Example: mount /dev/evms/sda1 /mnt/ifolder
Step 2:
Configure iFolder2.x from yast2.
  • In Linux server terminal type yast2 ifolder2.

  • For LDAP server configuration select "Remote System" and enter IP address of NetWare server, admin name with context and password.

  • Click next to continue.
Step 3:
Configure iFolder for coexistence mode. Give separate IP address. While configuring iFolder data location give path as /mnt/ifolder/ifolderdata (Create directory ifolderdata on iscsi partition) and start Apache.
  • Enter separate IP address for iFolder server. This IP address should be valid and should have valid DNS name.

  • Enter Netmask and DNS hostname for the same IP address.

  • Enter iFolder user data path as /mnt/ifolder/ifolderdata

      (Create ifolderdata directory on shared volume)

  • Click Next to continue and click Yes to start Apache.

  • Now iFolder2.x is configured.


Step 4:
Run umount /mnt/ifolder to Unmount iSCSI partition on first Linux machine.
Step 5:
Go to second Linux machine and repeat step 1 to step 4.
  • On second Linux machine do not create ifolderdata directory as it is already present on shared volume.
Step 6:
After Configuring iFolder on both the machine , remove IP address entry for iFolder server from this file "ifcfg-eth-id-xx.xx.xx.xx.xx.xx " on both the Linux servers.
  • vi /etc/sysconfig/network/ifcfg-eth-id-xx:xx:xx:xx:xx:xx

  • (Where xx:xx:xx:xx:xx:xx represent hardware address of Network card)

  • Delete secondary IP address (Configured for iFolder) and netmask entry from this file.

  • Delete lines IPADDR_ifdr and NETMASK_ifdr from this file.

  • This two lines will be present at the end of file.

Important: These lines should be removed from this file because we want to bind to the secondary IP address of iFolder using cluster service. If this entry is not removed then binding to this IP address will give error as "IP address is already in use".

8.0 Configuring iFolder files on cluster volume

Login as the root user on the first Linux machine.

Step 1:
Create directory on Shared disk (iSCSI partition).
  • Mount the shared disk that was created (e.g. mount /dev/evms/sda1 /mnt/ifolder).

  • Create the following directory on shared disk(e.g. /mnt/ifolder).

  • mkdir -p etc/opt/novell/ifolder.

  • mkdir -p var/opt/novell/ifolder.

  • mkdir -p srv/www/ifolder.

  • The directories just created should be owned by the root user and the default group should be root.

  • All four directories should have permissions of drwxr-xr-x.

  • Populate the shared disk.

  • Copy the contents of /etc/opt/novell/ifolder to /mnt/ifolder/etc/opt/novell/ifolder.

  • Copy the contents of /var/opt/novell/ifolder to /mnt/ifolder/var/opt/novell/ifolder.

  • Copy the contents of /srv/www/ifolder to /mnt/ifolder/srv/www/ifolder.
Step 2:
httpd.conf – iFolder

Modify /mnt/ifolder/etc/novell/ifolder/conf/httpd.conf

  • Change all occurrences of /etc/opt/novell/ifolder to /mnt/ifolder/etc/opt/novell/ifolder

  • Change all occurrences of /var/opt/novell/ifolder to /mnt/ifolder/var/opt/novell/ifolder

  • Change all occurrences of /srv/www/ifolder to /mnt/ifolder/srv/www/ifolder

Step 3:
httpd_ifolder_unix.conf

Modify /mnt/folder/etc/novell/ifolder/conf/httpd_ifolder_unix.conf

  • Change all occurrences of /etc/opt/novell/ifolder to /mnt/ifolder/etc/opt/novell/ifolder

  • Change all occurrences of /var/opt/novell/ifolder to /mnt/ifolder/var/opt/novell/ifolder

  • Modify the iFolderServerDNSorIP key to be the DNS name or IP address of the cluster resource. This is a secondary IP address.
Step 4:
listen.conf – Apache (it's sort of incomplete)

Modify /etc/apache2/listen.conf

  • Make sure the main instance of apache is configured to only listen on the primary IP address, not all IP addresses and not the cluster resource IP address. For example: "Listen 10.1.1.1:80" and "Listen 10.1.1.1:443" are valid. "Listen 80" and "Listen 443" are not valid as they listen on all IP addresses.

  • Restart Apache if changes are made to listen.conf.
    (e.g. "/etc/init.d/apache2 restart" )

9.0 Creating cluster resource using iManager

Step 1:
Make sure the shared disk is unmounted on both the Linux server.
Step 2:
Open iManager of NetWare Server and click on cluster.
(iManager URL https://<IP address of NetWare server>/nps/iManager.html)
Step 3:
Click on object selector and select cluster object (clusternew).
Step 4:
Go to cluster option and click on New to create new cluster template.

Step 5:
Click next and template will ask to enter load script. Copy load script given below.

Load Script for cluster

#!/bin/bash
. /opt/novell/ncs/lib/ncsfuncs
# mount the file system.
exit_on_error mount -t ext3 /dev/evms/sda1 /mnt/ifolder
# add the IP address
exit_on_error add_secondary_ipaddress <IP address of iFolder>
#setting permissions for ifolder data directory
exit_on_error chown -R novlifdr:ifdrwww / mnt/ifolder/ifolderdata
# start iFolder
exit_on_error /usr/sbin/httpd2-worker -k start -f /mnt/ifolder/etc/opt/novell/ifolder/conf/httpd.conf ?DSSL
# return status
exit 0
Step 6:
Click next to continue. Next screen will ask for unload script. Copy unload script given below.

Unload script for cluster

#!/bin/bash 
. /opt/novell/ncs/lib/ncsfuncs 
PID_FILE=/mnt/ifolder/var/opt/novell/ifolder/run/ifolder.pid
# stop iFolder 
ignore_error killproc -p $PID_FILE -TERM /usr/sbin/httpd2-worker
# Wait for all children to terminate 
for ((wait=0; wait"120; wait++)); 
do 
if test -f $PID_FILE;
then
usleep 500000
else 
continue 2
fi
done 
sleep 10 
# del the IP address 
ignore_error del_secondary_ipaddress <IP address of iFolder server>
# umount the file system 
exit_on_error umount /dev/evms/sda1
# return status 
exit 0 

This will create iFolder template as shown below:

Step 7:
Create new cluster resource and add this template for this resource. Once this is done iFolder is ready on cluster.


Editor's Note: FYI, this template will be included in Open Enterprise Server SP1. NCS 1.8.1 in OES SP1 will provide this template, so the steps given to create an iFolder template won't be necessary with OES SP1.

  • Cluster option -> New -> Resource
Step 8:
Click Next and give name for cluster resource as cluster_vol and click on object selector. Select cluster object. This will display following screen. Select iFolder template which was created before.
Step 9:
Click next to continue and click Finish to complete the configuration. iFolder resource will be displayed as shown below.

Now iFolder is ready for cluster.

Step 10:
For both the Linux servers, configure iSCSI and NCS to start automatically after reboot.
  • On both the Linux terminals type

  • chkconfig iscsi on

  • chkconfig novell-ncs on

Step 11:
Restart both the Linux servers.
Step 12:
After restart cluster resource will come up and it will be attached to any one node. Once the cluster resource comes up, open iFolder admin page by giving seconday IP address which is configured for iFolder.
(URL https://<iFolder server IP address>/iFolderServer/Admin
  • Go to iFolder servers, there will be two iFolder server entries on this page.

  • Both the servers will have same IP address.

  • Delete both the entries and create new server with same IP address.

  • iFolder server Admin page -> Global settings -> iFolder servers -> delete all.

  • Click on add to add new server .Enter IP address of iFolder server port and secure port as shown below.

Step 13:
Create iFolder users for this servers.

Login to iFolder server from Windows /Linux iFolder client.
Step 14:
Migration of cluster volume form node 1 to node 2 can be tested manually using iManager.
  • Cluster Manager ->click on cluster resource -> Migrate

  • Select node and click OK.

  • In case of Failover cluster resource will be migrated automatically from node 1 to node 2 and vice versa.

Additional Information:


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell