Improving eDirectory Performance on Linux, Solaris, AIX, and HP-UX Systems

The following sections provide information about how you can improve the performance of eDirectory on UNIX systems:


Fine-Tuning the eDirectory Server

Novell eDirectory on Linux and Solaris uses a dynamically adjusted thread pool to service client requests. The thread pool is self-adjusting and delivers optimum performance in most cases. However, you can avoid the delay caused by starting up threads when there is a sudden load on the server by setting the following parameters in the /etc/nds.conf file.

Parameter Description and Recommended Settings

n4u.server.idle-threads

Minimum number of threads (regardless of activity)

The value of this parameter should be based on the average client load, in order to minimize the time required to produce new threads during normal client activity.

n4u.server.max-threads

Maximum number of threads

The value of this parameter should be based on the maximum number of simultaneous clients that need to be serviced, along with the following recommendations:

  • eDirectory requires a minimum of 16 threads
  • One thread for every 255 LDAP connection (Monitor Thread)
  • One thread for every four concurrent clients (Worker Thread)

n4u.server.start-threads

Number of threads that start when eDirectory starts

The value of this parameter should be based on the average client load, in order to minimize the time required to produce new threads during normal client activity.


Optimizing eDirectory Cache

Novell eDirectory uses persistent caching so that changes being made to a server are held in a vector. If the server crashes in the middle of changes, eDirectory will load faster and synchronize the changes in seconds when the server is brought back up. Novell eDirectory uses a rollback model with a log file to roll forward transactions in the event of a system failure.

eDirectory settings begin with 16MB of cache, 50% of which is allocated to block caching and the other 50% is allocated to record cache. After 15 minutes, eDirectory will modify its cache thresholds to initialize up to 51% of the available free memory for the cache, leaving at least 24 MB for the OS. This algorithm is used only if the host OS supports the call that enables you to determine the amount of free memory available.

You can optimize your eDirectory cache in the following ways:


Using a Fixed Amount of RAM for UNIX Systems

Although the above algorithm works well for Windows and NetWare, it does not work as well for UNIX systems. On UNIX systems, the free available memory reported by the OS will be less than other operating systems because of the way the UNIX OS uses free memory for internal caching of file system blocks, frequently run programs, libraries, etc. In addition to this memory allocation, libraries on UNIX normally do not return the freed memory back to the OS.

For these reasons, we recommend allocating a fixed amount of RAM to the cache.

Fix the amount of RAM for UNIX systems by doing one of the following:


Manually Creating a .ini File

  1. Create a file called _ndsdb.ini in the same directory that the eDirectory database files (DIB set) are located (usually in /var/nds/dib).

  2. Add the following parameters listed in to the _ndsdb.ini file:

    Parameter Description

    blockcachepercent=50

    Sets the percentage of cache that is allocated to caching database blocks.

    cacheadjustinterval=15

    Sets the time (in seconds) in which eDirectory will evaluate its utilization of free memory and adjust the overall cache size.

    cachecleanupinterval=15

    Sets the time (in seconds) in which eDirectory will write dirty cache blocks to disk.

    cache=16777216

    Sets the hard limit (in bytes).

Using Novell iMonitor

  1. Click Agent Configuration Agent Configuration button.

  2. Click Database Cache, then view the following information:

    Database Cache Information Description

    Maximum Size

    The maximum size (in KB) that the specified cache is allowed to grow. to

    Current Size

    The current size (in KB) of the specified cache.

    Items Cached

    The number of items in the specified cache.

    Old Versions Cached

    The number of old versions in the specified cache. Old versions of cache items are kept to maintain the consistency of read transactions in the database. In other words, if one thread is in a read transaction and another is in a write transaction, old versions of blocks modified by the writer are maintained on behalf of the reader. This is done so that the reader's results are guaranteed to produce a consistent view during the life of its transaction even though modifications are taking place during that time.

    Old Versions Size

    The size (in KB) of the old version items cached.

    Hits

    The number of times an item was successfully accessed from the specified cache.

    Hit Looks

    The number of items looked at in the cache before an item was successfully accessed from the specified cache. The hit-look-to-hit ratio is a measure of cache lookup efficiency. Normally, the ratio should be close to 1:1.

    Faults

    The number of times an item was not found in the specified cache and had to be obtained in a lower level cache or from the disk.

    Fault Looks

    The number of items looked at in the cache before it was determined that the desired item was not in the specified cache. The fault-look-to-fault ratio is a measure of cache lookup efficiency. Normally, the ratio should be close to 1:1.

  3. Choose from the following options:

    Option Description

    Dynamic Adjust

    Allows the eDirectory database to dynamically adjust the amount of system memory to be used for the cache based on the amount it thinks it needs and the parameters specified below.

    Cache Adjust Percentage

    The percentage of available memory allowed to be used for the record and block caches combined.

    Cache Size Constraints

    While dynamically adjusting, follow the specified constraints. Namely, use no less than the specified amount of memory for the cache and no more than the total amount of available memory minus the specified amount.

    Hard Limit

    The exact amount of system memory to be use for the cache.

    Cache Maximum Size

    The size (in KB) of the record and block caches combined.

    Block Cache Percentage

    The percentage of the system memory available for caching that should be allocated to the block cache. The remaining percentage will be allocated to the record cache.

    Cache Adjust Interval

    This interval applies only when Dynamic Adjust is set. It controls how often the cache size is adjusted, based on the specified percentage and constraints.

    Cache Cleanup Interval

    Controls how often unused old versions are removed from the cache.

    Cache Settings Permanent

    When this option is selected, any changes submitted through iMonitor will be made permanent, overwriting any previously saved settings or system defaults.

  4. Click Submit.


Setting Cache Parameters

By default, eDirectory uses dynamic cache. If you have sufficient RAM to increase the eDirectory cache size, you can increase the performance of eDirectory considerably for large databases by allocating more RAM to the eDirectory cache.

The parameters that are listed in the following table can be adjusted to enhance your eDirectory performance:

eDirectory Cache Parameter Description

blockcachepercent=value

Sets the percentage of cache that is allocated to caching database blocks. The default is 50.

cachecleanupinterval=value

Sets the time (in seconds) in which eDirectory will write dirty cache blocks to disk. The default is 15.

cacheadjustinterval=value

Sets the time (in seconds) in which eDirectory will evaluate its utilization of free memory and adjust the overall cache size. The default is 15.

cache=value

Sets a hard limit (in bytes) of memory that eDirectory can use for caching.

cache=leave:value

Specifies the minimum number of bytes to leave.

min:value

Specifies the minimum cache size in bytes.

max:value

Specifies the maximum cache size in bytes.

According to the algorithm, the default setting for Novell eDirectory is the following:

cache=dyn,%:51,min:16777216,max:0,leave:0

This indicates the following:

  • The minimum cache size is 16 MB.
  • There is no maximum limit.
  • Dynamically, up to 51% of available memory will be used.
  • 24 MB should be left for the OS.

    eDirectory operates with a hard limit of 16 MB, so that all applications are started and the system is stabilized.

You can also configure eDirectory to use a percentage of the total memory. To do so, specify the cache as shown below:

cache=hard,total,%:percentage_of_total_memory_in_bytes


Optimizing Bulkload Data

Bulkload performance using the Import/Convert/Export (ICE) utility can be affected by a number of items. The most common performance hits come from poor Disk I/O management and the allocation of insufficient memory for the Novell eDirectory cache.

If eDirectory is essentially the only application, you can set the eDirectory cache up to 80% of the total memory. All allocated cache will eventually be used. eDirectory performance on highly volatile data is improved with more cache.

IMPORTANT:  You should avoid setting the cache memory size above 40% of the total memory if the server is hosting services or applications other than eDirectory. The smallest tested cache size is 0 and the largest is 3 GB. Determining the proper cache size depends on the memory needs of other processes running on the same server, and on the amount of disk cache required. You should test a variety of cache sizes to find a good balance.

To optimize the bulkload performance, allocate a higher percentage of the eDirectory cache for block cache. We recommend setting a value of 80% for block cache. This can be reset after the operation is completed.

Using iMonitor is the quickest way to modify the blockcachepercentage parameter. To do so, follow the instructions in Using Novell iMonitor.


Optimizing LBURP Transaction Size

The LBURP transaction size sets the number of records that will be sent from ICE to the LDAP server during a single transaction. Increasing this value can improve bulkload performance, assuming that you have adequate memory and that the increase does not cause I/O contention.

The default transaction size is 25, which is appropriate for small LDIF files (fewer than 100,000 operations) but not for a large number of records. The LBURP transaction size can be set anywhere between 1 and 1000.


Modifying the Transaction Size

To modify the transaction size, modify the required value for the n4u.ldap.lburp.transize parameter in /etc/nds.conf.

In ideal scenarios, a higher transaction size ensures faster performance. However, the transaction size must not be set to arbitrarily high values for the following reasons:

  • A larger transaction size requires the server to allocate more memory to process the transaction. If the system is running low on memory, this can cause a slowdown due to swapping.
  • Make sure that the LDIF file is free of errors and that any entries already existing in eDirectory have been commented out. Even if a single error exists in the transaction (including cases where the object to be added already exists in the directory), eDirectory will ignore the LBURP transaction setting and perform a commit after each operation to ensure data integrity.

    See Debugging LDIF Files for more information.

  • LBURP optimization works only for leaf objects. If the transaction contains both a container and its subordinate objects, eDirectory will treat this as an error. To avoid this, load container objects first from a separate LDIF file or enable the use of forward references.

    See Enabling Forward References for more information.


Tuning the Solaris OS for Novell eDirectory

The following sections provide information about how to tune the Solaris kernel, network, and file system:

IMPORTANT:  Before you begin, make sure that you have applied the recommended patches to the Solaris OS. For more information, see "Installing or Upgrading Novell eDirectory on Solaris" in the Novell eDirectory 8.7.3 Installation Guide.


Tuning the Solaris Kernel

To optimize the performance of eDirectory on Solaris, set the following kernel variables in the /etc/system:

Parameter Description

set maxphys=1048576

Maximum number of bytes that can be transferred per SCSI transaction.

set md_maxphys=1048576

Maximum number of bytes that can be transferred per SCSI transaction if you are using disksuite, vol_maxio, or vxvm.

set ufs:ufs_LW=1/128_of_available_memory

Barrier for the number of outstanding bytes on a single file below which the condition variable on which other sleeping processes are toggled.

set ufs:ufs_HW=1/64_of_available_memory

Number of bytes outstanding on a single fail barrier value.

ctcp:tcp_conn_hash_size=8192

Number of connection hash entries that are allocated to quickly locate the kernel data structures that are associated with TCP connection. (This can be increased to 262144, based on the number of LDAP clients.)


Tuning the Solaris Network

You can enhance LDAP search performance using the Solaris ndd command. The following command syntax allows you to analyze and modify tunable parameters that affect networking operation and behavior:

ndd -set /dev/tcp variable_name variable_value

The recommended values for the variables are listed in the following table:

Parameter Description

tcp_conn_req_max_q: 1024

The "q" stands for queue, which is the completed socket holding pen where sockets remain until the application issues accept.

tcp_time_wait_interval: 60000

Sets (in this case lowers) the time wait interval.

tcp_xmit_hiwat: 64000

tcp_xmit_lowat: 64000

Adjusts the minimum and maximum TCP send window size.

tcp_slow_start_initial: 2

Adjusts the number of first transmission packets from 1 to 2.


Fine-Tuning the Solaris File System

Novell eDirectory performance on Solaris can be improved if the Solaris file system is adequately tuned, especially for bulk loading data into the directory. File system tuning for eDirectory is similar to tuning for a database. See the Sunworld* Web site for more information on the Solaris file system.