Setting Up Dynamic Storage Technology with Open Enterprise Server 2 (OES2)
Novell Cool Solutions: AppNote
By Joe Harmon
Digg This -
Posted: 31 Aug 2007
- Setting Up Dynamic Storage Technology
- Retaining Trustees with Linux Backups
- DST Policies
- Using Policies
- Backup and Restore with DST
Setting Up Dynamic Storage Technology
To understand Dynamic Storage Technology, we recommend starting with the following white paper:
Dynamic Storage Technology (DST) requires a primary and secondary volume. Typically, these volumes are set up as NSS volumes, but they can also be set up on another volume type (like Reiser) depending on your needs.The example below demonstrates creating both primary and secondary volumes as NSS volumes; however, it is more likely that you already have an NSS volume that can be used for either the primary or secondary volume. If this is newer storage that you are using to create the new volume, then you probably want to use it as your primary volume. If it is older storage then you may want to set it up as the secondary. It is really up to you.
The volumes will be created with iManager 2.7 which is installed with the OES 2 server. If iManager 2.7 was installed into an existing tree, then you may need a replica as well as a LUM-enabled admin (or admin equivalent) user in order to log in.
- To configure the volumes, log in to iManager (https://IPorDNS/nps).
- Once logged in to iManager, go to the Storage plugin.
If your device is not yet available for creating a pool, you may first need to initialize the device by selecting the server using the browse button (looks like a magnifying glass), selecting the appropriate device and pressing Initialize Disk.
- You should receive a message warning that performing this action will destroy any data, partitions, and pools on this device.
- Go back to step 2 and perform the same operation on any other disk that needs to be initialized.
- After the disk has been initialized, you should be able to create a pool on the device.
Under Storage section, click on Pools. The server should already be selected from when it was configured in the device section.
- Click on New pool and enter the name of the new pool.
In this example, we will use one pool per volume and just give the pool the same name as the volume.
- Choose the disk(s) you want to add to the pool.
If you have only one disk, you can use a portion of the disk for one pool and a portion of the disk for another pool.
- You should now see a DATA pool on the configuration screen.
- Click on New pool again.
In this case we will be creating a pool called ARCHIVE.
- Again, choose the appropriate device and click Finish.
- Both pools should now appear in the list.
- Now select Volumes under the Storage plugin and click New.
- Create a volume called DATA and click Next.
- Choose the desired Pool (DATA pool in this example).
- Next, choose the appropriate volume attributes and click Finish.
- Repeat the same steps for the ARCHIVE volume, choosing the ARCHIVE pool.
When complete, both volumes should be listed.
- Now that the volumes have been created, we need to set up the ARCHIVE volume as a shadow of the DATA volume.
Since the ARCHIVE and DATA volumes are both NSS volumes, they are already set up as NCP volumes; as a result, the Novell Client can see both volumes, but we want the Novell Client to see both volumes as one.
To accomplish this, we first need to keep the ARCHIVE volume from loading as an NCP volume by creating an ncp2nss.conf file in /etc/opt/novell and adding the following information to the file:
EXCLUDE_VOLUME <nss volume name> EXCLUDE_VOLUME ARCHIVE
- Once the configuration file is complete, reboot the server.
- Now access the Novell Remote Manager by going to https://IPorDNS:8009 and expanding the View File System section.
- Click on Dynamic Storage Technology Options and you should see the DATA volume but not the ARCHIVE volume. If you do see the ARCHIVE volume, recheck the steps above.
- Click on the Add Shadow link next to the DATA volume, and then click Add Shadow Volume.
- Add the path to the ARCHIVE directory and click Create.
In this case it is /media/nss/ARCHIVE.
- You should now see that the ARCHIVE path has been added as a shadow to the DATA path.
Retaining Trustees with Linux Backups
If you will be using a backup utility with DST, you may need to add an NSS attribute that allows eDirectory trustee assignments to be backed up and restored. OES 2 has a switch that enables this capability. If the application in question does a listxattr to get all the extended attributes, then they will get the NSS netware.metadata attribute when you have the /ListXattrNWMetadata flag set. The linux cp command implements this as well as other tools. Test any third-party applications to be sure.
In this example, the Tivoli 5.4 Linux client is used in conjunction with the /ListXattrNWMetadata attribute to backup and restore trustee assignments. Without this attribute being set, the trustee assignments are not backed up by Tivoli and, therefore, are not restored.
To set the attribute, complete the following:
- Edit the /opt/novell/nss/conf/nssstart.cfg file:
- Add the following parameter to the bottom of the file:
- Save the file.
- You can either reboot for this attribute to take affect, or you can set the attribute using nsscon (requires a user with root access).
The only default policy with DST is to move files from the secondary file system to the primary file system when files are modified. This assumes an existing data store with both active and inactive data. The goal is to make the existing data store the shadow of the newly created data store. This way, the data is migrated as users access their files. Active files are moved instantly to the new data store (primary) and inactive files are left on the old data store (secondary). This also allows data migration to take place without taking a large initial hit on bandwidth. However, this method also takes more time because the data is only migrated as the user modifies files.
If you want to schedule a time for moving the data all at once, you can set up a policy or use the scan feature on the inventory page. The policy feature schedules the move for a given time; the scan feature moves data immediately.Both are explained briefly below.
Using the Inventory Scan Feature
Since the example we gave was to create two new volumes, we used the Novell Client to copy over the initial data. The Novell Client uses the virtual view so it sees files from both the DATA (primary) and ARCHIVE (secondary) volumes at the same time. Whenever anything is copied or saved using the Novell Client, it is always saved to the Primary volume. This gives us the opportunity to show how the Inventory Scan Feature can move files from one location to another.
- Login to NRM on the server running DST by going to https://IPorDNS:8009.
- Expand View File System, select Dynamic Storage Technology, and click on the Inventory link next to the DATA (Primary) volume.
- This displays useful information including graphs and charts on the file types, size, and owner, as well as modification and access information. A couple of examples are provided here to show how the information might be helpful in making decisions about data migration and policy creation.
For example, we can see that if we had a policy to move every file accessed in the last six months, we could reduce the backup interval tremendously because we would be backing up much less data. This means we could migrate everything less than six months old to the Primary partition and then create a policy to run this same migration at a given interval. Since our test data is on the primary, we want to migrate anything that is older than six months to the secondary volume (ARCHIVE). You can do this by migrating individual data sets by clicking on the file links in the chart, or by using the Inventory Scan Feature at the bottom of the Inventory page.
- Start by selecting the option to Move selected files from primary area to shadow area.
- To move all file extensions, leave the search pattern set to *.*.
- Set the time stamp to Last Modified Time.
- Set the range to everything over six months old and press Start Scan.
- A live report of the migration will be displayed.
- Once the migration has finished, a report indicating the Entries Moved along with the number of files is also displayed.
- If we again look at the inventory chart again after the move, we can see that the Primary volume has no files that are older than six months and the shadow volume has no files newer than six months. The default policy will move any files that are modified from the secondary to the primary and we can now create a policy to handle any files that are older than six months.
The previous section demonstrates the ability to move data using the Inventory Scan feature. In order to maintain that data migration, we need to create a policy. Or, if we couldn't use the Inventory Scan Feature due to the need to schedule down time, we could just set up the policy and let it take care of the migration.
You can create multiple policies and have them apply to a specific volume or all shadow volumes.
- If you have not already done so, login to NRM on the server running DST by going to https://IPorDNS:8009.
- Expand View File System, select Dynamic Storage Technology, and click Create a new policy.
- Add a description.
- Choose the start and stop time..
- Choose the start date. Note, you may want this to occur right before the backup happens.
- Choose the Frequency..
- Select the volume.
- Set any desired exclude restrictions.
- Choose the time stamp, interval, and file restriction if desired.
- Click submit.
Below is an example of how we set up maintenance for the files we moved.
Backup and Restore with DST
Introducing DST helps simplify the backup process. Rather than backing up all of the data all of the time, you can concentrate on backing up important data frequently and other data less often. But how do you restore that data when it is being backed up from multiple locations? Below are some possible suggestions for working with primary/secondary copies of with a virtual view, but you must first ask yourself some questions:
- Do I restore the information to the primary and/or secondary volume directly?
- Should I restore the data using the virtual view?
- Was data lost because the user deleted the file or by some other means (i.e. hardware failure, software issues, etc.)?
- How much data needs to be restored?
- Copies are stored directly on the volume in question so there is no need for the information to be transferred again through policies.
- You don't have the performance hit that you would with the shadow view.
- Restoration size isn't going to be an issue since you are restoring to the proper volume rather than though the virtual view.
- The backup software sees both volumes through one view; as a result, you could do a restore of the primary, secondary, or both through this view and let any duplicates be handled by the backup software.
- It doesn't matter whether the data is on the primary or secondary volume; if both are restored, the user will have his data.
Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com