Novell Home

AppNote: Backup and Restore with DirXML in a Mixed Environment

Novell Cool Solutions: AppNote
By Phil Kelly

Digg This - Slashdot This

Posted: 12 Feb 2004
 

Phil Kelly
Technical Director
Infant Technology Ltd.
phil.kelly@infatech.com

Introduction

Directory services lie at the heart of most organisations, and as choices increase in this field, it is becoming more common for companies to employ more than one directory service to provide a full range of services to its users.

Probably the most common example of co-existence is where companies employ both eDirectory and Active Directory, selecting the strengths of both platforms and leveraging both environments to gain best of breed technologies on both sides.

A company may wish to utilise eDirectory for its openness, its excellent LDAP, DHCP and DNS services or because its ZENworks tools provide just the right amount of control and flexibility that the customer needs.

They may use Active Directory as their file and print service platform, as well as for Exchange provision, SQL services, SharePoint Portal-based intranet and any number of other Active Directory centric applications.

Novell's DirXML product has been developed to provide a synchronization engine between the two disparate directories, and forms a key element of the Nsure product line. DirXML is a rules-based system, based on XML, where objects created, modified or deleted are synchronized according to those rules to another (directory) location.

DirXML includes a feature rich toolset allowing filtering of attributes based on source and destination schema and powerful transformation, mapping and placement rules.

Probably the least well understood area of DirXML implementations is that of how to provide a consistent and functional backup and restore platform. This is a critical area of implementation that must be addressed due to the issues associated with providing services to both sides of the DirXML connection.

It is important that organizations understand why backup and restore services should be implemented at both ends of the connectors, and the issues involved in restoring objects in the event of a problem occurring.

DirXML: How does it Work?

DirXML has been developed as a bi-directional interface between disparate systems and directories. Amongst the currently supported systems that the drivers interface with are:

eDirectory/NDS iPlanet Directory Server
Active Directory IBM SecureWay Directory
Windows NT Domains       Innosoft Directory
Exchange Critical Path InJoin Directory      
Lotus Notes LDAP
GroupWise Delimited Text Files

DirXML leverages the flexibility and extensibility of eDirectory to synchronize new and modified data across different directories and systems. It assists administrators and organizations to maintain a consistent data set across their systems by ensuring that relevant data is synchronized according to simple rules created to define the location of objects within the target system(s) and any transformation or mapping that must take place in order to maintain data integrity on the target system.

Synchronization is achieved by monitoring eDirectory and the target system(s) for events such as object creation, modification and deletions. Filters are used to define which objects and attributes of objects take part in this process.

Once an object fits these filters, it is submitted to the DirXML engine and is taken through a number of rules and style sheets that define how the object should be formatted, where it should be placed in the destination system and how the fields should be mapped between the two schemas.

Once this has occurred, the data is passed from the DirXML engine to the DirXML driver responsible for the target system, and the driver is responsible for contacting the destination and managing the action to be undertaken.


Outline of the DirXML workflow.

DirXML includes a limited number of command elements that describe the operational functions of the environment. These elements include <add>, <delete>, <add-association> and <modify>. Other elements are beyond the scope of this application note.

User Creation in eDirectory

When an object (ie user) is created in eDirectory and the object matches those rules laid out in the configuration of the DirXML driver, the DirXML engine sends an command to the Subscriber shim. The Subscriber shim then requests that the application (Active Directory in this case) adds an entry to its own system.

<add class-name="User" src-dn="\Darth Vader"> 
     <association>6023834</association> 
     <add-attr attr-name="cn"> 
          <value>Darth</value> 
     </add-attr> 
     <add-attr attr-name="Surname"> 
          <value>Vader</value> 
     </add-attr> 
     <add-attr attr-name="Given Name"> 
          <value>Darth</value> 
     </add-attr> 
     <add-attr attr-name="Telephone Number"> 
          <value>0207-000-0000</value> 
     </add-attr> 
</add>

The driver now passes the request to the external application (Active Directory) to perform the requested (<add>) operation.

The driver uses the account that was entered during the creation of the driver as the security context to undertake the operation. When the operation is complete, the driver returns an <add-association> event to the DirXML engine, informing the engine to add an association between the existing eDirectory object and the Active Directory object created as requested:

<add-association dest-dn="\Users\Darth Vader" dest-entry-
id="6023834">{BC3E7155-CDF9-d311-9846-0008C76B16C2}</add-association>

When the object has been created in the destination directory (in this case Active Directory), DirXML requests the GUID of the object and passes that back, to be used as the key attribute that governs the association between the two objects.

When this is done, DirXML flags the object's DirXML association as Processed.

Modifying an Object

Similar to the <add> command, when an object is modified, such as by adding a value to a field, a <modify> command is raised and passed to the driver by the DirXML engine.

How does DirXML Synchronize Objects?

DirXML employs a process to monitor those areas of eDirectory and Active Directory that are designated within the driver rules as participating in the synchronization process.

When an object is added that matches the various filters and contains the necessary attributes that fit with these rules, the appropriate commands (<add>, <modify>, <delete> etc) are raised and the engine and the driver transfer information back and forth between eDirectory and Active Directory.

Example DirXML Implementation

Many organisations will implement DirXML simply to provide synchronization services between two directories containing user and group information, as shown in the example below:


ConsoleOne, showing user and group objects.


Active Directory Users and Computers showing the same objects, synchronized
to their eDirectory counterparts with DirXML

The associations between the two entities are stored in eDirectory (where else?!) and can be seen under the object by choosing the DirXML tab under the object in ConsoleOne. This tab displays the Driver Object that generated the association, the Associated Object ID and the State of the association.


DirXML Tab of synchronized object in ConsoleOne

The Associated Object ID maps directly to the objectGUID attribute of the synchronized object in Active Directory. For confirmation of this value, use adsiedit, included on each Windows 2000 Server CD under \Support\Tools\2000RSKT.MSI:


Adsiedit confirms the objectGUID of the
synchronized object, displayed as an OctetString.
In this case the value is:

0x7b 0xcc 0x28 0xa6 0x4e 0xd1 0x93 0x46
0x80 0xd8 0x06 0x05 0x0b 0x33 0xa0 0xf6

What Happens when an Object is Deleted?

Of course, objects can be deleted on both sides of the DirXML connector for many reasons, and administrators must recognize the problems this will cause.

If the object is deleted from eDirectory first, that object is immediately flagged for a <delete> event in DirXML and at the next opportunity (usually within seconds of the object being deleted) it will be deleted from Active Directory also. If the object is deleted in AD, the same happens in reverse.

eDirectory Backup and Restore Basics

As stated in Novell's Cool Solution notice "Restoring NDS from Tape Backup: Proceed with Extreme Caution" (see resources at the end of this AppNote for further details), issues can and do occur when restoring eDirectory from tape.

Unlike the file system, eDirectory objects are referenced and linked to each other. They can't or only partially work without them. Tape backup products treat the eDirectory database like a file system, with containers being directories and objects being files.

This works great for backups, but when it comes to a restore, major headaches are likely to occur. As the Cool Solutions article points out, many concepts working for the file system just don't apply for a database, and after all, eDirectory is a very advanced database.

Tape restores of tree parts or full trees almost always result in incomplete objects, missing associations, invalid security equivalences or unknown objects.

The trouble is, in most cases the objects look fine in the administration tools! But a closer look reveals the incomplete data (see later in this article!).

While the more obvious associations like group memberships might seem easy to recover - if remembered by the administrator - many internally managed links are not even exposed in the administration consoles and therefore additional problems can occur in the background.

The only way to fully protect an eDirectory database is to protect the individual objects and all meta information in an object oriented way.

Common Backup and Restore Platforms for eDirectory and Active Directory

Selection of the appropriate tools for use in any directory services environment, DirXML included, is of paramount importance to the integrity of the installation.

Amongst the most widely used backup and restore solutions in eDirectory environments are:

Product Company
Backup Exec 9.1 for NetWare 9.1 VERITAS Software Corporation
BrightStor ARCserve Backup for v9 for NetWare   Computer Associates International, Inc.
DeTroubler, eDirectory Release future gate
eMBox Novell

Toolset Choices

The added complexity brought to an eDirectory environment when DirXML is introduced reduces the effectiveness of standard backup tools such as Backup Exec and BrightStor ARCserve; additional granularity of restoration is required as potentially object restoration with standard, SMS-based tools can have far reaching effects.

In reality, only two products exist that allow administrators to restore individual (or multiple) eDirectory objects. These products each have their own individual strengths and weaknesses.

Computer Associates BrightStor ArcServe 9

ARCserve 9 is a full blown backup and restore application for file systems, agents, and eDirectory also. ARCserve has existed and has supported NetWare platforms for many years and provides a superior service performing file and workstation agent-based backup services. It also provides backup and restore services to eDirectory and also includes limited facilities for restoration of individual eDirectory objects.

The process for restoring an individual object in ArcServe 9 begins with using the BrightStor ARCserve Manager to locate the object, and selecting it for restore, as shown in the following image:


Choosing an object to restore in ARCserve 9

Once the object (or objects) has been selected the administrator can recover the object from the backup of their choice, and once completed, ARCserve's Job Status function will confirm that the job is complete and its status (success/failure, etc).

If successful, the object has been restored to its original location within eDirectory. If the job status is marked as 'Failure' then further investigation is required to ascertain the reason and resolution for this failure.

Once ARCserve has restored the object, it is visible in ConsoleOne.

At first sight, the object would appear to be perfectly restored, and viewing the attributes of the object would suggest that it is a fully functioning object.

As the next two images detail, the object itself believes that it has group memberships, but subsequent checking of the group objects highlights that they have not been given that user object as a member, and therefore, the user object is not given membership of the group.


The user is successfully restored to eDirectory.


Restored user object has been restored with group membership.


Group does not include restored user.

This failure may not be too difficult to overcome for individual objects (however, file and directory assignments, ZENworks application associations, BorderManager privileges and a host of other attributes are not restored correctly either) but if your environment is complex, and multiple objects require restoration, it is better to employ a tool which does the job completely rather than in half measures.

eMBox; Novell Inc

Released with eDirectory 8.7.0, eMBox is being marketed as the new mechanism for backing up eDirectory.

In a DirXML environment however, eMBox has one major failing in that firstly it does not provide any granular restoration service. Therefore in the event of a restore, tools such as ArcServe and Backup Exec provide similar functionality.

DeTroubler, eDirectory Release

DeTroubler, eDirectory Release is a dedicated eDirectory backup and restore application that gives online, immediate restoration of individual (and multiple) objects and their attributes, containers and structures and complete eDirectory trees.

DeTroubler does not backup file systems or other services but only concentrates on eDirectory. It has a Windows GUI interface and excellent scheduling and management facilities.

It also does not suffer from the limitations associated with other backup software, including ARCserve, in that all attributes and values are restored correctly together with trustee assignments on local and remote servers.

The remainder of this Application Note uses DeTroubler in its examples due to the added limitations in the ARCserve product detailed above.

A Backup and Restore Methodology that Works

Most administrators would agree that the limitations of standard toolsets in an eDirectory environment can cause considerable issues should a restore operation ever be required. Similarly, Active Directory backup tools suffer from the same types of problems.

Microsoft has released tools such as ntdsutil with Windows 2000, and it is possible to restore objects using this tool and ntbackup but these restore operations are by no means online or transparent to the user population; servers must be taken down and restarted in directory restore mode, the operation must be undertaken then the server rebooted as normal.

Tools do exist however, that provide complete eDirectory and Active Directory coverage, and it is necessary that these tools are implemented in order for this coverage to include the DirXML service to provide enterprises with a complete wall-to-wall backup and restore platform.

Accidents Will Happen!

The following scenario has occurred time and time again within IT departments everywhere:

An administrator notices that somehow, a number of objects have disappeared from the eDirectory. The number of objects is not important; it could be one object or it could be hundreds or thousands.

Each of these objects had group members or memberships, ZENWorks application associations, login scripts, file trustee assignments, BorderManager privileges, and a whole host of other assignments in the eDirectory.

Not only that, but because of the DirXML association, the objects have also been deleted automatically from Active directory.

Of course, the AD objects also had their own assignments; NTFS rights, SQL login rights, Exchange mailboxes; the list and possibilities are literally endless!

The objects could have been deleted by an eDirectory administrator, or an administrator of Active Directory, but somehow they've been deleted.

Restoring Objects

Let us now examine the restoration process to bring objects back to full functionality in all three platforms (eDirectory, DirXML and Active Directory)

Restore the Object(s) to eDirectory

Firstly, we use DeTroubler to locate the object(s) to restore, as well as the version of the object to restore from the backup history.

Select the object to be restored in the DeTroubler interface and choose Operations then Restore Objects.


Selecting the object to restore, and the version to restore, in DeTroubler

This will now begin the restoration process for the object(s) that were chosen. DeTroubler will recreate the object exactly in the state it was when the backup was taken, including all application associations, file and directory assignments, group memberships and any other privileges that are defined. Note that unlike ARCserve, both sides of each assignment are restored, so no further work is required for the object to function as expected.

When the restoration is complete, DeTroubler will confirm that the restore has completed successfully:


The object is successfully restored using DeTroubler

Once the restoration is successfully completed by DeTroubler, the object exists in eDirectory in exactly the same state it was previously. However, the object will not be re-deployed to Active Directory via DirXML.

This is because when the object(s) were restored, and all attributes were restored, the DirXML associations were restored with their original value of 'Processed' and therefore DirXML assumes that no further work is necessary.


User object after restore in ConsoleOne. DirXML associations flagged as Processed

The user functions perfectly as an eDirectory entity; as can be seen in the image below, attributes and values are restored to their original state.


The object is recreated with all attributes in tact. The user object above displays its group memberships recreated. Also recreated are all ZENworks application associations and trustee assignments, whether directly or indirectly assigned.

Because the DirXML associations are set to 'Processed', if changes are now applied to the object, the synchronization still does not take place because the destination object does not exist in the target directory.


DSTrace detailing failure to synchronize with non-existent object in the source directory

The problem here is that the DirXML driver believes that the object has already been synchronized, and that all updates must be synchronized to an object in Active Directory with an objectGUID identical to the one stored against the object in eDirectory.

Simply recreating the object in Active Directory will not bring this back, because the objectGUID will be a different value and synchronization will not be possible.

It is possible to create a new user in Active Directory and to use adsiedit to locate the objectGUID, then to return to ConsoleOne and edit the DirXML association of the object to include the objectGUID value. This may be a valid action when restoring a single object, however, this process is fraught with danger, and the Active Directory account will then require reconfiguration to match the previous account - group memberships, NTFS rights, and so on.

A better approach is to utilise a dedicated recovery tool such as Aelita's ERDisk for Active Directory which offers a restore of single or multiple objects, including recreation of original objectGUID values, thereby ensuring that the DirXML synchronization process can be maintained.

Restore the Object(s) to Active Directory

The easiest way to restore the objects to the Active Directory platform and to ensure that we can bring the DirXML connection back successfully is to use Aelita's ERDisk for Active Directory product.

ERdisk for Active Directory performs the same actions that DeTroubler does, in that it allows administrators to restore individual or multiple objects to their original state, including all configuration attributes such as group memberships, etc.

Enter the ERDisk interface and browse to the objects to be restored, then follow the prompts to decide on the version to be restored, and other actions to take.


Begin the restoration process within ERDisk for Active Directory.

ERDisk will restore the objects as requested, and will ask if you wish to schedule an immediate replication cycle amongst domain controllers in the domain. If you have multiple sites, this is highly recommended.

Following restoration of the object in both eDirectory, with DeTroubler, and Active Directory, with ERDisk, both objects reference the same objectGUID value and so ongoing synchronization is possible and all operations are back to their original state.


Successful restoration of the object in ERDisk.

Sanity Check

We can demonstrate that the platform is fully returned to its original state by modifying the object and observing what happens in the DStrace, with the DirXML options set.

Open up a restored object in ConsoleOne. In the example below, we give the restored user object a title.


Editing the object.

Open up the object in ConsoleOne and assign a title to the restored user object.

Click on OK and switch to the DStrace screen.

The attempt to synchronize the object through the DirXML engine will appear, and if all operations in the restoration process have been carried out, the <modify> operation to add the title value will appear as below:


Successful re-synchronization of the object after restore.

The command to <modify> the object has been submitted from engine to driver. The driver has contacted the destination (Active Directory) application and the modification has been made. Then the driver returns the Status = success message back to the DirXML engine.


The changes made are visible in the destination directory

Confirmation is easy; simply open up Active Directory Users and Computers and select the object.

The user's Title attribute will show the correct value.

Conclusion

This document outlines a simple software platform and process for backup and restore of objects in a DirXML environment.

Without fully understanding the issues associated with these services in this environment companies are open to large-scale failures, magnified considerably as the number of directory connections is increased.

The software and processes outlined here seek to reduce and mitigate this risk and provide wall-to-wall coverage for both directory environments.

As we have seen in the field numerous times, companies are often slow to implement relevant technologies when undertaking DirXML projects, preferring to maintain their investment in tools which do not offer the same coverage.

The platforms detailed in this paper however are quick to implement and offer an excellent insurance policy against the loss of not one directory service but two or more, and the associated risk to loss of production for the organisation as a whole.

Resources

Additional reading on the subjects covered in this AppNote:

Restoring NDS from Tape Backup: Proceed with Extreme Caution
A Cool Solutions article advocating care when restoring eDirectory. Common sense stuff really, but someone had to say it!
Find it at:
http://www.novell.com/coolsolutions/nds/features/tips/t_restoring_tape_edir.html

DeTroubler for NDS
Cool Solutions Cool Tools
Find it at:
http://www.novell.com/coolsolutions/tools/1462.html

Vendor Web Sites and Contact Details

Mail Address: future gate
Rehkamp 10b
21521 Dassendorf b. Hamburg
Germany
future gate
1220 Dorchester Ave
Gwynn Oak, MD, 21207
USA
Telephone: +49 4104 9629 697 +1 866 211 5542
Facsimile: +49 4104 9629 699 +1 661 761 9121
Mail Address: Aelita Software Corporation
6500 Emerald Parkway
Suite 400
Dublin
Ohio 43016
USA
Aelita Software Europe
Merlin House
Brunel RoadTheale
Reading
Berkshire
RG7 4AB
Telephone: +1 614 336 9223 +44 (0)1189 026 690
Facsimile: +1 614 761 9620 +44 (0) 1189 026 698


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell