Novell Home

Monitoring Change Cache During Batch Update on a Replica Server

Novell Cool Solutions: Feature
By Subbu K.K.

Digg This - Slashdot This

Posted: 10 Apr 2003
 

eDirectory can appear to be slow and sluggish when bombarded with a large batch of create, modify or delete operations. This is because of the time taken to process and synchronize all the incoming changes.

For a replicated directory service like eDirectory, an incoming change request involves many housekeeping operations:

  1. Record the change in a transaction log for possible rollback in case of failure or abort.
  2. Recorded in a change log (Change Cache) for the partition involved for propagating it to other peer servers in the replica ring.
  3. Apply the change to the local DIB.
  4. Skulk the change to all peer servers in the ring.
  5. Once the change is processed, mark it for deletion from the change log. (aka Purging)
  6. Chase all references to the changed object from other servers and notify them about the change. (aka Backlink Processing)
  7. Physically remove entries from the change log after all the above operations are completed. (aka Obit Processing)

When a large batch of update is being processed, the change log can really get big and processing the changes in the background can take a long time. For a replica server to remain responsive and for the replica ring to converge quickly, it is important to keep the size of the change cache within manageable limits of the cache and disk i/o capacity. During a batch update, the replica server will be mostly write bound on the disk and the change cache will keep growing. If the change cache grows to a size where it outruns the capacity of the Entry Cache, then the response time of the server will degrade rapidly. At this time, suspend the updates and let the server process the accumulated changes. After a batch update, the server will be mostly read i/o bound on the disk as the change cache gets depleted. You can monitor the progress of the housekeeping operations by monitoring the change log size of all the replicas stored on the server.

You can do this manually through iMonitor web pages:

  1. Connect to the replica server
  2. Click on Agent Configuration
  3. Click on Partitions
  4. On each Partition listed, click on Change Cache.
  5. The number of items in the Change Cache for that partition is listed in "Subordinate Count:" at the top of the table.
  6. Note down this count.
  7. Repeat for each partition and total the counts.
  8. Click on "Agent Configuration" and then on "Database Cache"
  9. Note down the number of items in entry cache.

On UNIX/Linux, you can run the following script to do all this automatically during a batch update. This script will also load the server so poll the server with a gap of at least ten minutes.

------ count-change-cache.sh -------
1 #!/bin/sh
2
3 SERVER=${1:-192.168.1.101}
4 AUTH=${2:-admin.novell:secret} # use AUTHFILE for production trees
5 AUTHFILE=/etc/security/tree-$SERVER.pwd
6
7 getpage() {
8    if [ -r $AUTHFILE ]; then
9     lynx -dump -auth=`cat $AUTHFILE` "http://$SERVER/$1"
10   else
11    lynx -dump -auth=$AUTH "http://$SERVER/$1"
12   fi
13 }
14
15 for DN in `getpage nds/agent/partitions/data | awk -F= '/object\?dn=\/OU/ {print substr($0, index($0, "=")+1)}'`
16 do
17 count=`getpage "nds/changecache/data?dn=$DN" | awk '/Subordinate Count/ {print $3}'`
18 echo $count $DN
19 done | awk '{sum+=$1;print}END {print sum, "total"}'
20 getpage "nds/agent/data?config=CacheCtl" | awk '/Items Cached/ {print $3, "Items in Entry Cache"}'
--------------------------------------


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell