About the eDirectory Backup eMTool

The Backup eMTool provides hot continuous backup of the eDirectory database on an individual server. You can back up eDirectory on your server without closing the database, and you still get a complete backup that is a snapshot of the moment when the backup began. This feature means that you can create a backup at any time and eDirectory will be accessible throughout the process. (Hot continuous backup is the default behavior---you can specify a "cold" backup with the database closed, if required.)

The new backup also lets you turn on roll-forward logging to keep a record of transactions in the database since the last backup, so you can restore a server to the state it was in at the moment before it went down. You must turn on roll-forward logging for servers that participate in a replica ring, so that you can restore a server back to the synchronization state that the other servers expect. If you don't, when you try to restore from your backup files you will get errors and the database will not open. Roll-forward logging is off by default. For more information, see Using Roll-Forward Logs.

The new Backup eMTool does not back up all the objects in eDirectory at once; just the partitions on an individual server. This allows for better restore of an individual server and faster backups than the legacy TSA for NDS^® backup. (The legacy TSA for NDS backup still works as documented in eDirectory 8.6; both the TSA for NDS and the new backup can be used if necessary.) For a comparison, see What's Different about Backup and Restore in eDirectory 8.7.3?.

The new eDirectory backup tool must be used in conjunction with file system backups to put the eDirectory backup files safely on tape. Novell has partnered with several leading providers of backup solutions. For a list, see NetWare Partner Products: Backup, Restore, & Recovery.

On NetWare, you also might need to use the eDirectory backup tool in conjunction with backups of file system rights. For more information, see Preserving Rights When Restoring File System Data on NetWare.

In iManager, you can use all the features except cold backup, unattended backup, and advanced restore options, as explained in Using Novell iManager for Backup and Restore. All backup and restore tasks including unattended backups can be done using the eMBox Java command line client, as explained in Using the eMBox Client for Backup and Restore.

For a description of the backup and restore options in iManager, see the online help. For a description of the eMBox Client options, see Backup and Restore Command Line Options.

For a description of what the tool does during a restore, see Overview of How the Backup eMTool Does a Restore.

The eDirectory Backup eMTool is part of the eMBox tool set. The eMBox is a service that is installed on the server as part of eDirectory.

The Backup eMTool comprises the following files:

For a description of the format for the backup files and log files that the Backup eMTool creates, see Format of the Backup Log File and Format of the Backup File Header.

IMPORTANT:  The restore verification process is backward compatible only with eDirectory 8.5 or later. If you want to use the new backup and restore on servers that participate in a replica ring, make sure you upgrade all the servers in the replica ring to eDirectory 8.5 or later. (See also Restore Verification Is Backward Compatible Only with eDirectory 8.5 or Later.)

What's Different about Backup and Restore in eDirectory 8.7.3?

In previous versions of eDirectory, backup and restore was focused on backing up the tree, object by object.

The new Backup eMTool in eDirectory 8.7 introduced a completely new focus and new architecture. It's server-centric, not tree-centric; you back up the eDirectory database on each server individually. It's much faster than the legacy TSA for NDS backup.

The legacy TSA for NDS backup tool can still be used to back up the tree, although we encourage you to use the new backup.

Backup of server-specific information has been implemented using the Backup eMTool. See Changes to Server-Specific Information Backup (NetWare Only).

For more comparison information, see the following table.

Overview of How the Backup eMTool Does a Restore

Before restoring, you need to collect all your backup files by following the instructions in Preparing for a Restore. When you direct the Backup eMTool to begin the restore through iManager or the eMBox Client, the process is done by the Backup eMTool as follows:

1. The DS Agent is closed.
2. The active DIB (Data Information Base) set is switched from the DIB set named NDS to a new DIB set named RST.

   (The existing NDS database is left on the server; if the restore verification fails it will once again become the active DIB set.)

3. The restore is performed, restoring to the DIB set named RST.
4. The DIB set is disabled.

   The login disabled attribute is set on the pseudo server, preventing the DS Agent from being able to open using this DIB set.

5. The roll-forward log settings are reset to the default.

   This means that after a restore, roll-forward logging on the server is always set to off, and the location of the roll-forward logs is reset to the default.

   (If you want roll-forward logging turned on for this server, you must plan to re-create your configuration for roll-forward logging after a restore, to make sure it is turned on and the logs are being saved in a fault-tolerant location. After turning on the roll-forward logs, you must also do a new full backup.)

6. Verification of the restored RST database is performed.

   The server attempts to verify the consistency of the data that has been restored. It does this by contacting every server that it shares a replica with and comparing the transitive vectors.

   The output from this verification process is printed in the log file.

   If the transitive vector on the remote server is ahead of the local vector, then data is missing from the restore, and the verification fails.

   Here is an example of the information that's recorded in the log file if verification fails for one of the replicas, showing the transitive vectors that were compared:

   Server: \T=LONE_RANGER\O=novell\CN=CHIP 
      Replica: \T=LONE_RANGER\O=novell 
         Status: ERROR = -6034 
            Local TV             Remote TV 
            s3D35F377 r02 e002   s3D35F3C4 r02 e002 
            s3D35F370 r01 e001   s3D35F370 r01 e001 
            s3D35F363 r03 e001   s3D35F363 r03 e001 
            s3D35F31E r04 e004   s3D35F372 r04 e002 
            s3D35F2EE r05 e001   s3D35F2EE r05 e001 
            s3D35F365 r06 e003   s3D35F365 r06 e003 
   

   For more information, see Transitive Vectors and the Restore Verification Process.

7. If verification is successful, RST is renamed to NDS and the login disabled attribute is cleared so it becomes the active eDirectory database on the server. If verification fails, the RST DIB is not renamed, and the active DIB set is set back to NDS.

   If verification fails, see Recovering the Database If Restore Verification Fails for how to recover the server. (It's possible to force the RST database to be activated and unlocked using advanced restore options, but this is not recommended unless suggested by Novell Support.)

Format of the Backup File Header

The backup files contain a header that you can read to learn important information such as

• The filename of the backup file when it was created.

  This is helpful if the filename has been changed since the backup was created.

• The current roll-forward log at the time of this backup.

  If this is the last backup in the set you are restoring from (such as the last incremental backup in a set of one full backup and three incremental backups), this helps you because it indicates the first roll-forward log that you need for a complete restore.

• The replicas this server held.

  This is helpful if you did not have the placement of your replicas documented. If you experienced a disaster in which many servers were lost, the list of replicas shown in the backup file header might help you decide which servers to restore first.

• The names of the files that were included in the backup as specified in a user include file.
• The number of files in the backup set for that backup.

The header of the backup file for each individual backup is in XML format. Immediately following the header is the backup data from the database in binary code. (Because of the inclusion of binary data at the end of the file, parsing the file would give errors, but the XML header complies with XML standards.) In cases where the backup spanned more than one file, the header information is included in each file in the set.

WARNING:  When opening a backup file, just view the header---make sure you don't try to save or modify the file, or it might become truncated. Most applications can't save the binary data correctly.

The following is the DTD for the XML header. (The DTD is included as part of the header in the backup file as well, for your reference.)

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?> 
<!DOCTYPE backup [ 
<!ELEMENT backup (file|replica)*> 
<!ELEMENT file (#PCDATA)> 
<!ELEMENT replica EMPTY> 
<!ATTLIST backup version CDATA #REQUIRED 
    backup_type (full|incremental) #REQUIRED 
    idtag CDATA #REQUIRED 
    time CDATA #REQUIRED 
    srvname CDATA #REQUIRED 
    dsversion CDATA #REQUIRED 
    compression CDATA "none" 
    os CDATA #REQUIRED 
    current_log CDATA #REQUIRED 
    number_of_files CDATA #IMPLIED 
    backup_file CDATA #REQUIRED 
    incremental_file_ID CDATA #IMPLIED 
    next_inc_file_ID CDATA #IMPLIED> 
<!ATTLIST file size CDATA #REQUIRED 
    name CDATA #REQUIRED 
    encoding CDATA "base64" 
    type (user|nici) #REQUIRED> 
<!ATTLIST replica partition_DN CDATA #REQUIRED 
    modification_time CDATA #REQUIRED 
    replica_type (MASTER|SECONDARY|READONLY|SUBREF| 
    SPARSE_WRITE|SPARSE_READ|Unknown) #REQUIRED 
    replica_state (ON|NEW_REPLICA|DYING_REPLICA|LOCKED| 
    CRT_0|CRT_1|TRANSITION_ON|DEAD_REPLICA| 
    BEGIN_ADD|MASTER_START|MASTER_DONE| 
    FEDERATED|SS_0|SS_1|JS_0|JS_1|MS_0|MS_1| 
    Unknown) #REQUIRED> 
]>

The following table explains the attributes in the DTD.

The following is an example of a backup file header from a Windows NT server:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?> 
<!DOCTYPE backup [ 
<!ELEMENT backup (file|replica)*> 
<!ELEMENT file (#PCDATA)> 
<!ELEMENT replica EMPTY> 
<!ATTLIST backup version CDATA #REQUIRED 
    backup_type (full|incremental) #REQUIRED 
    idtag CDATA #REQUIRED 
    time CDATA #REQUIRED 
    srvname CDATA #REQUIRED 
    dsversion CDATA #REQUIRED 
    compression CDATA "none" 
    os CDATA #REQUIRED 
    current_log CDATA #REQUIRED 
    number_of_files CDATA #IMPLIED 
    backup_file CDATA #REQUIRED 
    incremental_file_ID CDATA #IMPLIED 
    next_inc_file_ID CDATA #IMPLIED> 
<!ATTLIST file size CDATA #REQUIRED 
    name CDATA #REQUIRED 
    encoding CDATA "base64" 
    type (user|nici) #REQUIRED> 
<!ATTLIST replica partition_DN CDATA #REQUIRED 
    modification_time CDATA #REQUIRED 
    replica_type (MASTER|SECONDARY|READONLY|SUBREF| 
    SPARSE_WRITE|SPARSE_READ|Unknown) #REQUIRED 
    replica_state (ON|NEW_REPLICA|DYING_REPLICA|LOCKED| 
    CRT_0|CRT_1|TRANSITION_ON|DEAD_REPLICA| 
    BEGIN_ADD|MASTER_START|MASTER_DONE| 
    FEDERATED|SS_0|SS_1|JS_0|JS_1|MS_0|MS_1| 
    Unknown) #REQUIRED> 
]> 
 
<backup version="2" backup_type="full" idtag="3D611DA2" time="2002-8-19'T10:32:35" srvname="\T=MY_TREE\O=novell\CN=DSUTIL-DELL-NDS" dsversion="1041081" compression="none" os="windows" current_log="00000003.log" next_inc_file_ID="2" number_of_files="0000001" backup_file="c:\backup\header.bak"><replica partition_DN="\T=MY_TREE" modification_time="s3D611D95_r1_e2" replica_type="MASTER" replica_state="ON" /><replica partition_DN="\T=MY_TREE\O=part1" modification_time="s3D611D95_r1_e2" replica_type="MASTER" replica_state="ON" /><replica partition_DN="\T=MY_TREE\O=part2" modification_time="s3D611D95_r1_e2" replica_type="MASTER" replica_state="ON" /><replica partition_DN="\T=MY_TREE\O=part3" modification_time="s3D611D96_r1_e2" replica_type="MASTER" replica_state="ON" /><file size="190" name="C:\WINNT\system32\novell\nici\bhawkins\XARCHIVE.001" encoding="base64" type="nici">the data is included here</file> 
 
<file size="4228" name="C:\WINNT\system32\novell\nici\bhawkins\XMGRCFG.KS2" encoding="base64" type="nici">the data is included here</file> 
 
<file size="168" name="C:\WINNT\system32\novell\nici\bhawkins\XMGRCFG.KS3" encoding="base64" type="nici">the data is included here</file> 
 
<file size="aaac" name="C:\WINNT\system32\novell\nici\nicintacl.exe" encoding="base64" type="nici">the data is included here</file> 
 
<file size="150" name="C:\WINNT\system32\novell\nici\NICISDI.KEY" encoding="base64" type="nici">the data is included here 
</file> 
 
<file size="4228" name="C:\WINNT\system32\novell\nici\system\Xmgrcfg.ks2" encoding="base64" type="nici">the data is included here 
</file> 
 
<file size="168" name="C:\WINNT\system32\novell\nici\system\Xmgrcfg.ks3" encoding="base64" type="nici">the data is included here 
</file> 
 
<file size="1414" name="C:\WINNT\system32\novell\nici\xmgrcfg.wks" encoding="base64" type="nici">the data is included here 
</file> 
 
</backup>

After the header, the binary data for the backup of the database is included in the backup file.

Format of the Backup Log File

The eDirectory Backup eMTool keeps a log that shows a high-level view of Backup eMTool activity, containing information about previous backups. The log file contains a history of all backups, records backup start time and end time, and contains information about possible errors during the backup process. This file is appended with each backup. It is also placed in a location you specify.

It is useful for reviewing whether unattended backups were successful. The success or failure and the error code are displayed on the last line.

The Backup eMTool log file also gives the ID of backups that have been done, which helps you gather the correct set of full and incremental backup files for a restore. The first four lines are duplicates of information in the header of the backup file.

Also recorded in the log file are other files that were included in the backup of the database, such as the files you specified in an include file.

For a restore, it will record the included files that were restored.

The following are two examples of log file entries:

|==================DSBackup Log: Backup================| 
Backup type: Full 
Log file name: sys:/backup/backup.log 
Backup started: 2002-6-21'T19:53:5GMT 
Backup file name: sys:/backup/backup.bak 
Server name: \T=VIRTUALNW_TREE\O=novell\CN=VIRTUALNW 
Current Roll Forward Log: 00000001.log 
DS Version: 1041072 
Backup ID: 3D138421 
Starting database backup... 
Database backup finished 
Completion time 00:00:03 
Backup completed successfully 
 
|==================DSBackup Log: Restore================| 
Log file name: sys:/save/doc.log 
Restore started: 2002-7-19'T19:1:34GMT 
Restore file name: sys:/backup/backup.bak 
Starting database restore... 
Restoring file sys:/backup/backup.bak 
Database restore finished 
Completion time 00:00:15 
Restore completed successfully

Using DSMASTER Servers as Part of Disaster Recovery Planning

If you have a multiple-server environment and want to plan for recovery after a disaster in which all your servers are lost, you can use DSMASTER servers as part of the plan for your tree.

The Backup eMTool is used to back up each server separately; it is server-centric, not tree-centric. However, if you create DSMASTER servers, you can use Backup eMTool functionality specifically to back up your whole tree structure. An example of this strategy is outlined in Scenario: Losing All Servers in a Multiple-Server Environment.

When restoring after a disaster, one of the main concerns is how to avoid restoring replicas of the same partition that are inconsistent with each other. If you lose roll-forward logs for your servers as part of a disaster, you won't be able to restore all your servers to the same moment in time. Without the roll-forward logs, the replicas you have in your backups are inconsistent with each other and would cause problems if they were all restored and brought into the tree together. (The restore verification process is designed to help prevent these problems; by default a restored eDirectory database will not open after the restore if it is inconsistent with the other replicas.)

You can use DSMASTER servers to help you prepare for this issue, by creating a master copy of your tree that you could use as a starting point.

To use DSMASTER servers to help prepare for a disaster:

• Plan your replicas so that you have one server that contains a replica of every partition in your tree, so a copy of the whole tree is in the eDirectory database on one server (or, if your tree is large, you can use a couple of key servers). This kind of server is often called a DSMASTER server. The replicas on the DSMASTER server should be master or read/write replicas.

  NOTE:  If a couple of key DSMASTER servers are used instead of just one, keep in mind that ideally each DSMASTER server should have a unique set of replicas of partitions. There should be no overlap between them, to avoid inconsistencies between the replicas when restoring after a disaster.

  If your servers were lost in a disaster, you would not have access to the most recent roll-forward logs for restoring because roll-forward logs are saved locally on the server, so all the DSMASTER servers probably could not be restored to the same moment in time. If the same replica were held on two DSMASTER servers, the two copies would probably not be identical and would cause inconsistencies in the tree. So, for disaster recovery planning it's best to not have the same partition replicated on more than one DSMASTER server.

  For general information on replicas, see Replicas.

• Back up these DSMASTER servers regularly to create a backup copy of your tree. You might want to take extra precautions for storing the backups of DSMASTER servers as part of your disaster recovery plan.

If your tree is designed this way, in the event of a disaster you could get your tree structure up and running again quickly by restoring just that one server (or small group of key servers) and making sure the replicas it holds are designated as the master replicas.

After your tree structure is responding again, you could then move to the task of restoring other servers that were lost, using just the full and incremental backup files. Because you don't have the roll-forward logs, the verification of the restore process will fail for these other servers. To bring them back into the tree, you would remove them from the replica ring, change all their replica information to external references using DSRepair, and then re-add the replicas to the servers using replication from the copy on the DSMASTER server. These steps are documented in Recovering the Database If Restore Verification Fails.

If a disaster occurs in which you lose many servers but not all, the issues with replicas will probably be complex, and you should contact Novell Support.

Transitive Vectors and the Restore Verification Process

A transitive vector is a time stamp for a replica. It is made up of a representation of the number of seconds since a common specific point in history (January 1, 1970), the replica number, and the current event number. Here's an example:

s3D35F377 r02 e002

In the context of backup and restore, it's important because the transitive vector is used to verify that the server restored is in sync with the replica rings it participates in.

Servers that hold replicas of the same partition communicate with each other to keep the replicas synchronized. Each time a server communicates with another server in the replica ring, it keeps a record of the transitive vector the other server had when they communicated. These transitive vectors allow the servers in a replica ring to know what information needs to be sent to each replica in the ring to keep all the replicas synchronized. When a server goes down, it stops communicating, and the other servers don't send updates or change the transitive vector they have recorded for that server until the server starts communicating again.

When you restore eDirectory on a server, the restore verification process compares the transitive vector of the server being restored to the other servers in the replica ring. This is done to make sure that the replicas being restored are in the same state that the other servers expect.

If the transitive vector on the remote server is ahead of the local vector, then data is missing from the restore, and the verification fails. (For example, data might be missing because you did not turn on continuous roll-forward logging before the last full or incremental backup, you did not include the roll-forward logs in the restore, or the set of roll-forward logs you provided for the restore was not complete.)

By default the restored eDirectory database is not opened if it is inconsistent with the other replicas.

For an example of the log file entry when transitive vectors don't match, see Overview of How the Backup eMTool Does a Restore.

For a description of compatibility issues that could cause the restore verification to fail, see Restore Verification Is Backward Compatible Only with eDirectory 8.5 or Later.

For information on what to do if the restore verification fails, see Recovering the Database If Restore Verification Fails.

Restore Verification Is Backward Compatible Only with eDirectory 8.5 or Later

The restore verification process is backward compatible only with eDirectory 8.5 or later. If the server you are restoring participates in a replica ring with a server running an earlier version than eDirectory 8.5, the restore log will show a -666 error (incompatible DS version) for that replica. This does not indicate whether the replicas are out of sync; it merely indicates that the restore verification was unable to compare the transitive vectors because the version of eDirectory was earlier than 8.5.

By default the database will not open because the restore verification was not completely successful. However, you can use your best judgement in this case; if the only error was from an 8.5 server, and the other servers verified successfully, it might be safe to open the database, using the override restore option in the eMBox Client.

Another option might be to remove the server with the older version from the replica ring, and try the restore again.

For more information about the restore process and transitive vectors, see Overview of How the Backup eMTool Does a Restore and Transitive Vectors and the Restore Verification Process.

Preserving Rights When Restoring File System Data on NetWare

On NetWare only, restoring file system rights (also called trustee assignments) is dependent on the object that is the trustee being present in eDirectory. Because of this relationship, you need to use caution when restoring eDirectory and file system data on NetWare, to preserve file system rights.

If you restore eDirectory before restoring file system data, file system rights should be preserved when file system data is restored. However, you should be aware of the issue. Test for potential problems and take preventive action if necessary.