Novell Home

Bulkloading eDirectory Objects

Novell Cool Solutions: Tip
By Subbu K.K.

Digg This - Slashdot This

Posted: 1 Oct 2001
 

If you ever have the need to load a huge number of eDirectory objects in a short period of time, you know how daunting that task can be. Here are some suggestions to speed up the process. If you've got other ideas to add, please let us know.

How to stage a 1 million object tree quickly on a Server with just 1GB of RAM? Here's a tip. Please note that there are many parameters that influence the overall time taken. This covers only some of them.

A 2m object tree with default object (without password) will consume at least 2.4GB (or 600,000 blocks of 4KB each). If you can keep the block write time below 20ms, a flushing operation will take 12,000 secs or about 3.3 hours. Since we also write transaction logs during this time, the actual time could be between 3.3 to 6.6 hours. You may want to plug-in the actual numbers (size of object, block write time in ms) into these equations to estimate the time in your specific case.

Most of my experiences have been on Solaris and Linux, but I do think the following will hold good on NetWare also:

  • Modify operations are cache hungry. If you reserve 256MB for the base OS and file blocks, you are left with only 744MB of RAM. I would try with the following _ndsdb.ini
    • cache=780140544
    • blockcachepercent=98
  • Suspend all indexes (sub-string indexes hog the most resources).
  • Use a transaction size of about 200 in ICE.
  • Get a RAID expert to tune the system to keep the block write time as small as possible. Striping helps, mirroring doesn't.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell