Novell Home

DSClone - A New Tool for Disaster Recovery

Novell Cool Solutions: Feature
By Paul McKeith

Digg This - Slashdot This

Posted: 22 Mar 2005
 

Here is a helpful explanation of how DSClone (a relatively new and sometimes misunderstood feature of eDirectory) can be used to recover eDirectory, especially in large environments.

About DSClone

DSClone is usually used for creating replicas on newly installed servers, where the data size and network bandwidth are at odds with each other. See http://www.novell.com/documentation/edir873/edir873/data/acavuil.html#aky13ak for details on using dsclone. It is a feature intended to be used through iMonitor.

However, you can also use DSClone as a key tool in your disaster recovery process, as this article explains.

How DSClone Works

Any science fiction fan has probably seen a movie or read a book where someone clones himself and sends the clone off to work while he reaps the benefits. Somehow, the clone has all the accumulated knowledge of the original. In a way, that's what DSClone can do for eDirectory. It makes it possible to build a server in a tree with all the eDirectory objects you need, and then make one or more duplicates of that server with all its "accumulated knowledge" of objects.

When DSClone is run on a server, it uses that server as the clone source to create a new NCP server object in its tree - the clone target. The new object is added to all the replica rings to which the clone source server was a member. This will immediately cause the other servers in the rings to report -625 synchronization error messages, but this is expected and normal. Next, DSClone makes a copy of the clone source's DIB files to a specified location on the clone source. These files must then be manually copied/moved to the clone target. Once the DIB files on the clone target are opened by eDirectory, it recognizes that it is a clone and modifies the DIB files to make the DIB files its own.

Example

Suppose you need to create an eDirectory tree with 1 million objects for an LDAP application, and it needs to be built as soon as possible. You follow normal eDirectory design rules and create a new tree with three physical servers, so you can maintain three copies of every partition. You need to make sure your tree can survive a site disaster, so you put one of the servers across a WAN link.

Now that the tree is built, you use an LDIF file to add the 1 million objects to one of the servers in the tree. Then you wait - not only for the objects to load into the target server, but also to synchronize to the other two replicas in the tree. One of the servers is across a slow WAN link, so this will take a long time to complete. That long wait on synchronization can be eliminated with DSClone.

Here's the DSClone approach:

  1. Install the current and same version of eDirectory on all the servers, but only one server is actually in the tree.
  2. Partition the tree according to your tree design.
  3. With only one server in the tree, add the 1 million objects to the tree/server.
  4. Once the initial load in done, use that server as the clone source to create the second server which is not yet in the tree.
  5. After you have performing a quick health check, use one of the two existing servers as a clone source to create the third server in the tree. Note that you never add the clone target servers to the tree in the traditional manner. DSClone will add these servers to the tree for you.

DSClone will create the new server object in the tree and provide you with a set of DIB files to be moved to the clone target. Make sure that when you install eDirectory on the clone target server, it's the same version and release level as the clone source. When the clone target opens these DIB files, it contacts the Master replica and knows that it is a clone. At this point, it automatically modifies the local DIB files so they now belong to the new server. The new server now has a copy of the same partitions as the clone source and will properly participate in eDirectory synchronization. This process is simply repeated on the third server to complete our example.

If objects are added, modified, or deleted on the clone source after the running DSClone, but before the new server comes online, normal eDirectory synchronization will take care of the differences. The normal process to add a server to the tree is skipped. The new server is treated as if it were a normal healthy and consistent server that was simply "downed", with all its objects, at the time of the clone. Of course, major changes such as partition operations should not be possible until the new server is up and participating in normal eDirectory synchronization processes.

Guidelines

The traditional rules and guidelines for partition operations still apply when using DSClone. Here are some points to keep in mind:

  • Like a partition operation, be sure to complete the one clone and perform a quick health check before starting another clone.
  • Do not add new servers to any affected replica rings when you are cloning a server.
  • Do not clone a server and leave that server down for an extended period of time. Doing so will cause existing servers to potentially maintain a large number of changes that will have to synchronize when the clone target is finally brought online.
  • A
  • s with any eDirectory environment, servers must be offline for the shortest time possible to maintain a healthy tree. Since the clone process is very fast, there is no need to clone a server until the target server has been built and is ready to receive the cloned DIB files.

DSClone for Disaster Recovery

As shown in the example above, DSClone is primarily a way to "copy" eDirectory object and partition information from an existing server in a healthy eDirectory tree to a "new" server in the same tree. In this context, a "new" server can truly be one that has never existed in the tree before, or a new server that is replacing one that was in the tree but was suffered a catastrophic failure.

When DSClone is used for disaster recovery, it does not restore the failed server to its previous state. Instead it provides a way to re-build the failed server. Its greatest benefit is to rebuild a failed server that has one or more partitions with a large number of objects. Even with the best server hardware and a high-speed network, rebuilding a server by manually adding 10 partitions each with 500,000 objects will each take hours or even days to complete. DSClone can reduce the time to minutes.

DSClone provides an efficient means to rebuild a failed server with a large DIB. It is not an eDirectory backup/restore solution. For eDirectory backup and restore, consider using the eMBox back-up and restore feature. For more information, check out the eDirectory documentation and/or TID 10093373: eDirectory Backup and Recovery using eMBox/eMTool - http://support.novell.com/cgi-bin/search/searchtid.cgi?/10093373.htm.

DSClone is not a comprehensive disaster recovery solution. There are aspects of a disaster that DSClone does not address. It is only one tool in your disaster recovery toolbox. It is possible that DSClone will not be the best method to recover eDirectory data. This is particularly true for traditional File/Print services tree where the number of objects involved is relatively low.

DSClone and Certificate Authority

DSClone can rebuild a failed server that was the tree's Certificate Authority (CA). DSClone itself will not restore the CA to fully functioning status. In this case, the CA object must be deleted and restored using a backup/export of the CA. Regardless of whether you ever plan to use DSClone, if you have not yet exported your CA to a file, stop reading this and do so now ... I will wait. It only takes a minute and will save you hours if you ever need it. If you need help check out the eDirectory documentation and/or TID 10071751: - Backing up and Moving the Tree Certificate Authority - http://support.novell.com/cgi-bin/search/searchtid.cgi?/10071751.htm.

If the failed server was the tree CA, it was likely also the Security Domain Infrastructure (SDI) key server, also known as the "tree key" server. If you are saying to yourself "That's OK, I never heard of that so I must not have one" ... watch out. The SDI key server was installed automatically so that all the servers in the tree share the same tree key. Features such as Universal Password, NMAS, and Novell Secret Store use the tree key to encrypt sensitive information stored in eDirectory. If your SDI key server fails and is not recovered properly, this could lead to very odd behaviors that are difficult to troubleshoot. For more information about SDI and what it means to you check out TID 10083941: How does Novell implement a Security Domain Infrastructure? - http://support.novell.com/cgi-bin/search/searchtid.cgi?/10083941.htm.

DSRepair -RC vs. DSClone

A frequently asked question is this: Why do I need DSClone when I have DSRepair -RC? Aren't they the same thing?

No. DSREPAIR -RC is a point-in-time archive of a local DIB and is server-specific. It can be used to recover a failed server but never to build a new server. As you probably know, eDirectory is very dynamic, so point-in-time backups have limited usefulness - just like data backups to tape. Plus, the distributed nature of eDirectory can make such point-in-time snapshots useless. Nevertheless, with the guidance of Novell Technical Support, a DSRepair -RC archive can be used to recover from some disasters. Still, this is usually a last-resort option, because using an old DIB is fraught with danger. If an old DIB is used improperly, the best case would be that you have inconsistent replicas in your tree. The worst would be a catastrophe of Biblical proportions. OK, not really - but you get the idea that DIB sets are not intended for eDirectory backup purposes.

DSClone allows you to prepare the eDirectory tree to use a copy of the local DIB of the clone source so it can be: a) restored to a crashed server, or b) copied to a new server as in the example above. DSClone takes a point-in-time snapshot of a server but unlike DSREPAIR -RC, it does so in such a way that the entire tree is aware that the clone process has started. With this awareness, the tree servers maintain changes to eDirectory objects since the clone initiation so the changes are synchronized to the new clone target when it is started. The tree can never "know" when a snapshot is taken using DSREPAIR -RC, so changes since it was completed are not "tracked". Think of it as a way to get a large DIB onto a new server without waiting for replication to complete - just use DSClone to clone an existing healthy server to create a new or replacement server. DSClone and eDirectory work together to make all the necessary changes so the new server works properly.

What DSClone Does NOT Do

When a server is added to a tree using traditional installation methods, the server is not simply added to the tree like it was back when NDS was first introduced. Today, a typical server also has other services that create eDirectory objects for the server. For example, Server certificates, Volumes, SAS, httpserver, and LDAP Server objects are all created by the services responsible for them. Since DSClone copies a DIB file set from an existing server, the source server objects are unique and cannot be "cloned" for use with the new NCP server object. Such objects are not created by the DSClone process.

If you have recently used NWCONFIG on NetWare to remove a server from a tree and re-install it into the same or another tree, you are familiar with the objects affected. The same processes to recreate these objects are the same as those used to complete the DSClone process. When using DSClone to add new servers to a tree, this is a trivial concern, because the objects never existed in the first place, and nothing was using them. These objects simply need to be created using the proper tools.

When using DSClone for disaster recovery, the implications are more significant. The original NCP server object must be manually deleted (to keep the same identity) and then recreated by DSClone. On NetWare, the server references can be backed up before the NCP server object is deleted, by using the Xbrowse utility, but this is not always successful. On other eDirectory platforms, objects that reference the original server object will have to be deleted and re-created. Luckily, most of these objects are not used on non-NetWare platforms, so they are of little concern.

With these limitations, it may seem that DSClone is more trouble that it is worth. In some cases that may be true. It is most useful and most appropriate for large-scale eDirectory implementations primarily used as "LDAP" servers. This is quite a different role for eDirectory File/Print trees - here, objects are highly distributed among many servers and partitions with total server object counts are relatively high. When dealing with trees that have 10s of millions of objects, the traditional recovery methods break down.

Choosing a Clone Source

The server you choose to clone will determine the partitions that will be "copied" to the clone target. There is no way to clone individual partitions, only servers. Suppose you are replacing a server that has partitions A, B, and D. Your clone source should be a server with those partitions. If that server has additional partitions, those too will be copied. But after the cloning process is complete, you can simply remove the extra partitions as needed. The clone source can have either Master or Read/Write replicas. If possible, choose the server with Master replicas as the clone source.

Creating a Clone

A cloned DIB file set can be created with the originating server, either online or offline. The offline method requires eDirectory to be brought down on the clone source. Online mode allows eDirectory to remain running throughout and after the cloning process. Online mode may take longer to complete than offline mode.

Preparing the Clone Target

If the clone target is to use new or different server hardware, make sure that it has the same version and patch level as the clone source. This may require that you add the server to a temporary tree and apply patches. If you use a temporary tree, be sure to have a unique tree name on the network. The server name can be the target server name as long as it is also unique on the network.

Building the clone target may take longer than the clone process itself, so build the target server first - it is best to perform the clone once the target server is ready to accept the cloned DIB files. If your tree/server is mission critical, it may be a good idea to have a server pre-installed and waiting. To save on costs, an option often used is to build the server in a temporary with a spare hard disk, then remove the hard disk and put it on a shelf instead of having the entire server sit idle. If you can boot your servers to a SAN, use the SAN as the "shelved hard disk". Another option is to use server-imaging software to keep a vanilla build available. Just be sure to update the image as you upgrade/patch your production environments. The clone source and clone target must be running the same version of eDirectory.

Online Method

Here are the steps to cloning a DIB fileset online:

1. Extend the schema of the tree. Make sure you do this, or an error will occur. Use dibclone.sch, which is present in the eDirectory installation. This will add the needed attributes for the iMonitor clone utility to operate. Platform-specific instructions are listed below.

NetWare

Use NWConfig (NWConfig.nlm > Configuration Options > Directory Options > Extend Schema). dibclone.sch is located in sys:\system\schema.

Windows

Use NDSCons.exe (in NDSCons.exe, load install.dlm, then click Install Additional Schema Files). dibclone.sch is located in C:\Novell\NDS.

Linux, Solaris, AIX, and HP-UX

Use ndssch. dibclone.sch is located in /usr/lib/nds-schema. For more information, see Using the ndssch Utility to Extend the Schema on Linux, Solaris, AIX, or HP-UX.

2. Create the clone DIB fileset.

3. Load the dsclone module on the server to be cloned. Platform-specific instructions are listed below.

NetWare

At the server console, enter dsclone.nlm.

Windows

In NDSCons.exe, select dsclone.dll, then click Start.

Linux, Solaris, AIX, and HP-UX

Add an "ndsclone" entry to the ndsmodules.conf file, then use the http://IP address:port/dhost page to load the Directory Clone Agent.

4. Run Clone DIB Configuration in iMonitor.

5. Click Agent Configuration > Clone DIB Set > Create New Clone.

6. Specify the fully qualified name of the server to be cloned and the file path where the cloned DIB files will be placed, then check the Create Clone Object and the Clone DIB Online boxes.

7. Click Submit. The NDS Clone object is created and the DIB fileset is copied to the specified destination.

8. Move the cloned DIB fileset onto the receiving server in the proper directory location.

9. Transfer the /etc/nds.conf file to the receiving server and update all the references in the file with the server name.

10. Run eDirectory on the cloned server. Make sure the master replica of the clone Server object is running eDirectory and is available. When eDirectory initializes on the cloned server, it communicates with the master replica so the final naming of the cloned server is resolved.

Offline Method

1. Extend the schema of the tree. Make sure you do this, or an error will occur. Use dibclone.sch, which is present in the eDirectory installation. This will add the needed attributes for the iMonitor clone utility to operate. Platform-specific instructions are listed below.

NetWare

Use NWConfig (NWConfig.nlm > Configuration Options > Directory Options > Extend Schema). dibclone.sch is located in sys:\system\schema

Windows

Use NDSCons.exe (in NDSCons.exe, load install.dlm, then click Install Additional Schema Files). dibclone.sch is located in C:\Novell\NDS

Linux, Solaris, AIX, and HP-UX

Use ndssch. dibclone.sch is located in /usr/lib/nds-schema. For more information, see Using the ndssch Utility to Extend the Schema on Linux, Solaris, AIX, or HP-UX.

2. Create the clone DIB fileset.

3. Run Clone DIB Configuration in iMonitor.

4. Click Agent Configuration > Clone DIB Set > Create New Clone.

5. Specify the fully qualified name of the server to be cloned, then check the Create Clone Object box.

6. Click Submit. The NDS clone object is created, eDirectory is brought down, and an error reports that eDirectory is locked.

7. Manually copy the *.nds, nds* and nds.rfl/*.* files to the Clone Server to a destination or media convenient for moving the set to the destination server.

8. Bring up eDirectory on the originating server. Move the cloned DIB fileset onto the receiving server in the proper directory location.

9. Run eDirectory on the cloned server. Make sure the master replica of the clone Server object is running eDirectory and is available. When eDirectory initializes on the cloned server, it communicates with the master replica so the final naming of the cloned server is resolved.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell