27.2 Tuning Cache Buffers for NSS on Linux

27.2.1 Understanding How NSS Uses Cache on Linux

NSS manages cache buffers on Linux using methods similar to those used in other Linux file systems such as Reiser, Polyserve, and XFS, with the exception of EXT.

For file data, NSS uses the Linux cache page manager to gain access to available memory in the system. There are some limits in place so that when copying large files, NSS does not starve other user applications for memory. This is similar to the cache handling used in NetWare.

For metadata, NSS uses kernel memory. NSS can use only a percentage of this space because other applications share this space. By default, NSS reserves a minimum buffer cache size of 30,000 4-KB buffers, which is about 120 MB of the kernel memory space. You can adjust the minimum number of buffers to be used by NSS with the MinBufferCacheSize parameter.

For a 32-bit machine, the kernel cache memory limit is 1 GB cache. Depending on what else is running, you might need to modify how much space you allocate for NSS.

For example, when running ravsui(8) for a pool verify or a pool rebuild, the utility needs contiguous space in kernel memory separate from the space allocated to the core NSS process. The larger the pool, the larger the space that is needed. On a 32-bit machine with a 1 GB limit, you might need to stop other processes temporarily to free up space so that the verify or rebuild process can run. You can optionally modify the amount of space used by the core NSS process by lowering the setting for MinBufferCacheSize to as little as 10000 4-KB buffers. When the verify or rebuild is done, you can change the setting back to its normal setting.

27.2.2 Setting the Minimum Number of Cache Buffers to Use for Kernel Memory

  1. Open a terminal console as the root user.

  2. Start nsscon(8). At the console prompt, enter

    nsscon
    
  3. Set the minimum number of cache buffers used by NSS on Linux. In nsscon, enter

    nss /MinBufferCacheSize=value
    

    where value is the number of 4-KB buffers to assign for NSS. The default value is 30000.

    The maximum setting is the amount of memory in KB divided by 4 KB. For a 32-bit machine the maximum setting is 250000 buffers.

27.2.3 Configuring High Memory Cache

Linux limits the low memory to less than 1GB on 32-bit machines, but places all physical memory in the low-memory area for 64-bit machines. By default, NSS on Linux caches its metadata blocks in the server's low memory. When the working set of metadata does not fit into low memory, performance is affected. In particular, metadata caching on servers with large numbers of files can create performance issues for servers that are low memory constrained as for 32-bit machines.

NSS for Linux supports metadata caching in high memory for servers that are low-memory constrained. This can greatly improve metadata reads on 32-bit systems with large numbers of files.

Configuring High Memory Cache Type and Size

Enter the following command in the NSS Console (nsscon):

nss /HighMemoryCacheType=[private | linux | none]

Set the type of use of User memory in 32-bit OES Linux as none, linux, or private. On 64-bit OES Linux, NSS does not use User memory at all.

Cache Type Options
private

Set up a separate user-mode cache in memory outside of the Linux cache. This is the default for 32-bit machines. By default, NSS takes 20 percent of the total high memory. You can modify this setting with the nss /PrivateCacheSizeInBlocks=value command.

However, If the number of Private Cache blocks is less than twice the NSS Cache blocks, then the default high memory cache type is linux. If the high memory is less than twice the NSS Cache blocks, then the default high memory cache type is none.

Private Cache is best used for dedicated file-servers. The Private Cache increases the performance of NSS, but reduces the amount of memory available to other file systems and tasks.

linux

Integrate caching into the regular Linux caches. This is the default for 32-bit machines where the number of Private Cache blocks is less than twice the NSS Cache blocks.

This can be a problem on a dual purpose file-server and application-server system, because memory-hungry applications can cause the file-system cache to purge completely.

none

Use the same algorithm as the initial release of OES 1 Linux, which is to try and cache everything in Kernel-mode memory. This is the only option on 64-bit OES Linux. This is the default for 32-bit machines if the high memory is less than twice the NSS Cache blocks.

nss /PrivateCacheSizeInBlocks=value

Sets the number of metadata blocks to cache for a 32-bit OES Linux machine where you have set the HighMemoryCacheType=private.

By default, NSS takes 20% of all high memory to use for the Private Cache. We recommend against giving NSS 100 percent of the high memory. The actual percentage you should allot depends on your deployment scenario. For example:

  • Which features of NSS are enabled--salvage, compression, and so on.

  • Use of really long filenames.

  • Use of extended attributes.

Default: 20 percent of all high memory blocks

Range: 0 blocks to the number of blocks available in high memory (total available high memory divided by 4KB)

Monitoring the High Memory Cache Use

To view the current settings for the high memory cache usage, enter

nsscon /status

View the following parameters in the Current NSS Status report:

  • Current NSS Status:
  • ... (parameters omitted)
  • Current Private Cache Size = 387680k (96920)
  • Private Cache Size Limit = 596364k (149091)
  • High Memory Cache Type = private
  • ... (parameters omitted)

The Current Private Cache Size indicates the amount of memory currently being used by the Private Cache. The Private Cache Size Limit shows the maximum amount of memory that the Private Cache can use. The High Memory Cache Type indicates the type of high memory cache that is being used as none, linux, or private.

27.2.4 Setting the Name Cache Size

The NSS Name Cache is responsible for caching the Name Tree information. This is the information that is read when you perform any kind of search by file or directory name. The Name Cache maps a name to a ZID (a unique file object ID). Directory listings do not do this as much as normal file opens that must resolve each name in the file path.

Use the NameCacheSize parameter to specify the amount of recently used Name Tree entries for files and directories that NSS caches. Each entry uses about 150 bytes of memory.

Increasing the maximum number of entries Name Cache entries does not necessarily improve the performance for getting directory listing information if NSS also needs to look up information about the file from a tree or structure outside of the name tree.

If you want to see how your name cache is performing, use the nsscon /NameCacheStats command.

nsscon /NameCacheSize=value

Specify the maximum number of recently used Name Tree entries for files and directories to cache. Name cache grows up to the specified limit. Unlike the file system cache, it does not take the maximum amount of memory allocated from the start.

Default: 100000

Range: 17 to 1000000