Resolving Container Audit Problems

This section describes solutions to potential container audit problems. These include audit trail overflow and synchronization of the container audit files to other NDS partitions, as well as recovery from catastrophic failures.

Audit Trail Overflow

Preventing Loss of Audit Data describes the potential for audit loss if the configured number of audit files are filled or disk space fills up and the audit trail is improperly configured.

Audit Options Configuration describes the three overflow configuration options for container audit trails: archive audit file; disable audited events; and disable event recording. The only option that prevents the loss of audit events (from audit overflow situations) is to disable audited events. With this setting, the server disables all audited NDS events when the current audit file has reached the Audit file maximum size or the server cannot write the current audit file (for example, out of disk space).

In this overflow state, any event that is preselected for auditing is disabled. For example, if logins are preselected for auditing, any attempt to log in to an object in the container (except for attempts by auditors of the container) would fail.

To recover from the overflow state, an auditor (with the Write right to the container Audit File object Audit Policy property) must reset the current audit trail.

1. Log in to the network as an auditor of the offending container.
2. If you want to save the oldest audit file and you haven't already backed it up, then copy the oldest old audit file to offline storage (for example, a file in the server or workstation or removable media).
3. Reset the current container audit file, as described in Reset Audit Data File. This rolls over the current audit file (to an old audit file), deleting the oldest old audit file, and initializes a new audit file.
4. If you want to save any audit files that you haven't already saved (including the newest of the old audit files), then copy those audit files to offline storage.

Consider the following suggestions to help prevent container overflow:

1. Perform frequent reviews of the status and size of the audit file.
2. If necessary, manually reset the audit file before it overflows.
3. Enable Automatic audit file archiving as described in Audit Options Configuration. Set the Audit file maximum size large enough and the Days between audit archives low enough that the audit file will not overflow. Use caution in setting these parameters to prevent destruction of audit data.
4. Don't over audit.

   WARNING:  If the audit trail for a container is full, the auditor's actions (for example, deleting data files, resetting the audit file) might not be audited for that container. In this case, you must keep a manual log of your actions for use when generating a complete history of actions performed on the server. You will be informed via a message from the server to your workstation when this occurs.

   When the audit trail is reaches its configured threshold, you will receive the following notification on your workstation screen:

   The audit overflow file for container  contname is almost full. Auditors must begin manual auditing now!

   When the audit trail is completely full, you will receive the following notification on your workstation screen:

   The audit overflow file for container  contname is full.

   To avoid missing this message, you must not issue the SEND /A=N or SEND /A=P commands (or if using Windows* and the NetWare User Tools, do not disable network warnings), as they would cause these messages to be suppressed.

Container Audit File Replication

Container audit files are replicated by NDS to the servers that hold replicas of the container object. That is, if container LAB1.ENGR.ACME is replicated by NDS onto three different servers, then the audit file for that container will also be replicated onto the same three servers. Replication of container audit files is automatic, and there is no way that you can tell the server to not replicate the audit file, other than to not replicate the audited container.

WARNING:  Replication of container audit files requires disk space on multiple servers. For example, if your container audit trail is configured for 16 audit files (1 current, 15 old) of 1 MB and the container is replicated on three servers, then the audit storage could require as much as 16MB on each server, for a total of 48 MB of disk space.

Records in replicated container audit trails are not necessarily stored in the same order, however, each replica of the audit file will eventually include nearly all of the audited events. In some rare cases (as described in Generating Container Audit Reports) records might be in some instances of the audit trail but not others.

Catastrophic Failure Recovery

This section describes what to do if you have a catastrophic failure, such as

• All copies of the Audit File object describing the container being audited are destroyed (perhaps because of a hard disk failure) and you need to recover the audit state to what it was before the failure
• Volume SYS: on one or more servers holding the audit data are destroyed

In addition, it explains how to handle planned upgrades, such as when a server is upgraded so that volume SYS: (where the container audit data is stored) is moved from a small disk drive to a larger disk drive.

There are several potential losses not addressed here:

• Loss of offline audit data. Your offline audit data (be it stored in server or workstation file systems, or on removable media) should be backed up frequently enough that its loss would not be catastrophic. Because different copies of the container can have different audit data in their audit files, it might be important to create offline copies of all instances of each container audit file.
• Loss of some, but not all, copies of the Audit File object describing the container audit trail due to failure of one or more servers holding an NDS partition. In this case, NDS will automatically use whatever copies are available. If a server configured for the partition is brought back online, it will automatically be updated with the Audit File object information.

There are two major catastrophic failures possible for external audit:

• Loss of all copies of the Audit File object describing the container audit trail. If all copies of the Audit File object are lost (for example, because there only was one copy, and the server it was on suffered a disk failure), then you might be able to recover the Audit File object from a backup of your Directory tree (presuming you have backed up your Directory tree). If so, then you can regain access to the existing online audit data. If not, then no access is possible to the online audit data. You must recreate the container audit trail using the procedures in Enabling Container Auditing (including selecting events, audit full actions, and so on).
• Loss of volume SYS: on any server containing the container audit data (for example, because of a disk failure). Because container audit files are stored in an inaccessible directory which cannot be backed up, loss of volume SYS: means that the online audit files (both the current audit file and any old audit files) are lost. You should use AUDITCON to perform regular backups of audit data to avoid loss of online audit data.

  If there is at least one other server with a copy of the container, then when the failed server comes back online it will automatically be updated, and container auditing will automatically resume. If there are no other servers with copies of the container, you must restore the container from an NDS backup and recreate the container audit trail using the procedures described in Enabling Container Auditing.

  WARNING:  If you restore a container from an NDS backup, it will come back without auditing. To avoid unaudited actions while you are configuring the audit system, you should take the server offline for the restoration process until the container audit has been reconfigured. To do this, disconnect the server from any networks it is connected to, and attach it to a protected LAN containing only a trusted workstation located in a secure location. Then restore the NDS container from the backup. Use the trusted workstation to run AUDITCON to re-enable container auditing, restoring the previous configuration. Finally, reconnect the server to the standard networks.

If you upgrade volume SYS: (for example, replacing it with a larger disk), that is equivalent to recovery from a catastrophic disk failure. To do an upgrade, you must first back up the old volume, and then restore it on the new disk. This loses all audit data. Therefore, before performing a volume upgrade, you should also back up all container audit data stored on that server. When the upgrade is complete, NDS will automatically cause auditing to resume for those containers with at least one other copy in the network.

Immediacy of Changes

When you modify the container audit trail configuration (for example, to change the maximum size of the audit file or the set of events to be recorded), the change is made both to the Audit Policy property of the Audit File object and to the header of the current audit file in the selected replica of the container.

Both changes will usually occur immediately. From the selected replica, the changes propagate to all other instances of the replica, which again usually happens immediately. However, the effect of the change might not be immediate if one or more of the servers holding the audit data are unavailable to receive the configuration change (for example, because they are down or the network has been split), even though the Audit File object can be modified.

In this case, the delay depends on how long it takes before the two servers can synchronize their NDS replicas.

In addition, if an auditor is performing audit trail management functions, changing the ACL will not affect the auditor's capabilities (either to increase or decrease them). An auditor's rights are recalculated every time he or she restarts AUDITCON and establishes access to an audit trail. To stop the auditor's actions immediately, you should break the auditor's connection to the server using the console MONITOR utility or the CLEAR STATION console command.