Developing exteNd Director Applications

CHAPTER 24

Using the Cache Coordinator

This chapter describes how to set up the exteNd Director Cache Coordinator for server cluster configurations. It has these sections:

For more information For background information, see Working with Data Caches.

About the Cache Coordinator

exteNd Director provides a Cache Coordinator for managing cached data in server cluster configurations. Specifically, the Cache Coordinator manages objects stored in the server-lifetime cache. (see Working with Data Caches). The Cache Coordinator runs on one machine in the server cluster.

How the Cache Coordinator works

You may store any objects in a server-lifetime cache holder. However, only the objects that are instantiated from data from a persistent store (such as a database) can be synchronized in the cluster environment.

All servers in a server cluster access the same persistence store. Any cached objects that are instantiated from the persistence store can be synchronized among application servers that are running in a clustered environment. Here's how it works:

When data (or an object) has been updated in the database, the cached object is updated accordingly in the affected server's lifetime cache holder. At this point the server's Cache Manager triggers a cache invalidation event that sends a notification to the Cache Coordinator.
The Cache Coordinator broadcasts a message to the rest of the registered server instances to tell them that this data has been changed.
After receiving the message, each server flushes its server-lifetime cache.
The next time this data is requested, each server instantiates the updated object by fetching it from the persistent store, and recaches it in its server-lifetime cache. This ensures that the cached objects are always up-to-date.
The Cache Coordinator also periodically verifies that each cluster server is running, removing from the cluster any server that has failed.

NOTE: Data is not replicated to different server instances. This means if you store temporary data in the server lifetime cache, the data will not be replicated to other server instances.

Triggering a cache invalidation event

If you want to cache an object that will automatically trigger an invalidation event, use EbiSrvLifetimeCacheHolder.putObjectInCache() and pass in EboState.UPDATE. Calling EbiSrvLIfetimeCacheHolder.removeObjectInCache() always triggers the cache invalidation event.

IMPORTANT: Do not remove objects from cache if those objects are not physically removed from persistent storage.

If you want to cache an object without triggering the cache invalidation event, use EbiSrvLifetimeCacheHolder.putObjectInCache() and pass in EboState.SILIENCE.

For more information, see EbiSrvLifetimeCacheHolder in the API Reference.

NOTE: exteNd Director provides built-in server-lifetime cache holders for many types of subsystem runtime data, as described in Built-in cache holders.

Reconfiguring the Cache Coordinator

By default, the Novell exteNd Installation Wizard installs the Cache Coordinator. (You can override this behavior using a custom installation). Here is a list of Cache Coordinator properties that you can reconfigure and where you can access each property.

Setting	Description	Editable in
Server Host	The name of the host machine the Cache Manager runs on. The default value is localhost. IMPORTANT: You must change the hostname from localhost to the actual host name for the Cache Coordinator server. The localhost entry will disable the cache invalidation feature.	FrameworkService config.xml
Server Port	RMI registration port the Cache Coordinator listens on. The default value is 54490. You must specify the same value when you create an exteNd Director project.	SilverCache.props FrameworkService config.xml
Application Identifier	UID for the exteNd Director application. This is created automatically in the exteNd Director Project Wizard when you select clustering. IMPORTANT: If you plan to use the shared library feature in a server cluster, each server must point to a single ConfigService.jar that includes the cluster properties.	Not editable
Start retry count	Number of times the server application instance attempts to connect with the Cache Coordinator before generating an error.	FrameworkService config.xml
Start retry interval	Number of seconds before each connection retry.	FrameworkService config.xml
Watcher sleep interval	Number of seconds the Cache Coordinator waits between attempts to contact the server application instance. If an application fails, the Cache Coordinator drops it from the cluster.	SilverCache.props
Reconnect sleep interval	How long after a connection failure the Cache Coordinator will wait before attempting to reconnect.	FrameworkService config.xml

Procedure To edit Cache Coordinator properties in config.xml:

In your exteNd Director project, open config.xml in the FrameworkService-conf directory.

This opens the XML Editor. Find the property you want and edit the value.

NOTE: The following shows the source view of the properties. You can also use the Graphical (default) view in the editor. Choose Graphical View or XML Source View in the editor.

  </property>
      <property>
          <key>com.sssw.fw.coordinator.Host</key>
          <value>myCMServer</value>
      </property>
      <property>
          <key>com.sssw.fw.coordinator.Port</key>
          <value>54490</value>
      </property>
      <property>
          <key>com.sssw.fw.coordinator.start.sleepInterval</key>
          <value>10</value>
      </property>
      <property>
          <key>com.sssw.fw.coordinator.Start.tryCount</key>
          <value>3</value>
      </property>
      <property>
          <key>com.sssw.fw.coordinator.reconnect.sleepInterval</key>
          <value>60</value>
      </property>

Save the file and deploy the project.

Procedure To edit properties in SilverCache.props:

Open SilverCache.props, located at:

installdir/exteNdDirector/bin/SilverCache.props

Find the property you want and edit the value:

  # This properties file contains properties for the
  # extendDirector Cache Coordinator
  # version 4.0 or later
  
  cc.port=54490
  cc.watcher.interval=60
  cc.logging.level=3

Save the file and run the Cache Coordinator, as described next.

Running the Cache Coordinator

Procedure To run the Cache Coordinator from the Windows programs menu:

Make sure the server cluster is running.
Go to Start>All Programs>Novell exteNd 5>Director>DirectorCacheCoordinator.

Procedure To run the Cache Coordinator manually:

Make sure the server cluster is running.
Run the Cache Coordinator:
- In Windows NT or Windows 2000, run the following command from the exteNd Director installation directory:
```
  bin\SilverCacheCoordinator.exe
```
- On Solaris, run the following command from the exteNd Director installation directory:
```
  bin/SilverCacheCoordinator
```

Procedure To run the Cache Coordinator as a service (Windows only):

Go to the Services panel.

In Windows NT choose Services from Control Panel>Administrative Services.

In Windows 2000 choose Services from Control Panel>Administrative Tools.
Select Silver cache coordinator.
Select Start to manually start the program, or select Startup and change the service to automatic so it will automatically run in the background when you start the server cluster.

Recovering from a Cache Coordinator failure

If a server in the cluster cannot connect to the Cache Coordinator, the server disables the cache invalidation function and reconnects to the Cache Coordinator. If the reconnect fails, you can flush the cluster server caches without restarting the servers. You can access each server in the cluster from the Director Administration Console (DAC) and use the Flush All option to:

Clean up the caches
Reconnect to the Cache Coordinator instance in the cluster

NOTE: This option does not apply to a server cluster running on the WebSphere application server. You must restart each server in the cluster.

If you restart the servers in the cluster, then flushing the cache is not necessary: the servers start with nothing cached.

For more information about accessing cache information using the DAC, see Using the General Configuration Section of the DAC.

Logging Cache Coordinator activity

exteNd Director provides logging facilities for Cache Coordinator activity. You can monitor cache invalidation messages sent from each server instance to the Cache Coordinator, and monitor activity on the Cache Coordinator itself.

To set up cache invalidation logging you need to do the following:

Update the logging level to 5 for each server instance and for the Cache Coordinator.
To make logging more legible, provide a server identifier for each server instance.

NOTE: A ServerID is provided automatically with some application servers. See Providing server identifiers.

For more information For general information about logging and logging levels, see Logging Information.

Updating the logging level for server instances and the Cache Coordinator

Cache invalidation logging uses Cache logging level 5.

You can set the log level two ways:

To change the logging level permanently Change the EboCacheLog.LoggingLevel property (in FrameworkService-conf /config.xml) to 5. Updating this property value will persist the change for subsequent server sessions.

For more information For information on editing this file, see To edit Cache Coordinator properties in config.xml:.

To change the logging level for the server session Change the logging level using the DAC. Making this change allows you to monitor cache invalidation messages sent from this server instance to the Cache Coordinator for the current server session.

For more information, see Using the General Configuration Section of the DAC.

Updating the logging level for the Cache Coordinator

You also need to update the logging level to 5 on the Cache Coordinator server. Use one of these methods:

To change the logging level permanently Change the logging.level property in SilverCache.props to 5. Updating this property value will persist the change for subsequent server sessions.

For more information For information on editing this file, see To edit properties in SilverCache.props:.

To change the logging level for the server session Run the Cache Coordinator in debug mode by adding an option switch:

  SilverCacheCoordinator -l 5

This will provide information about how many application instances are registered with the Cache Coordinator.

TIP: For more information about command options, run:

  SilverCacheCoordinator -?

Providing server identifiers

In order to allow better logging and have a readable server ID for each registered server instance, certain application servers provide an attribute that clearly identifies each server instance. As an alternative you can use the -D server startup option

Support for exteNd Application Server There is an attribute available from ServletContext called com.novell.appsrv.servlet.application.host:port that automatically provides a different Server ID for each server instance.

Support for WebSphere Application Servers There is an attribute available from ServletContext called com.ibm.websphere.servlet.application. that automatically provides a different ServerID (host:port#) for each server instance.

Server startup option You can use the +D server startup option to provide your own n identifier for each server instance. Specify a value for the ServerID attribute, as shown in this example:

  Silverserver +Dcom.novell.afw.ServerID=myID

where myID represents a unique identifier.

NOTE: This option overrides any identifiers specified elsewhere.

The ServerID attribute is combined with a Cache Manager ID that is an automatically generated UUID for each server restart.

NOTE: If a ServerID is not specified only the Cache Manager ID is used. Since this value is difficult to read in the context of logging, it is not recommended.

About the logging messages

Logging messages are generated on the sever instances and the Cache Coordinator whenever a server is started and when server instances generate cache invalidation messages.

Server startup logging

A message is generated on each serve instance when it attempts to register with the Cache Coordinator at startup. Here is what the logging message looks like on a server instance:

  EboCacheLog|5|11/11/03|15:20:09:113|<Connect> to the cache coordinator @ michigan:54490...
  EboCacheLog|5|11/11/03|15:20:09:154|<Cache Coordinator Service rmi URI> is: 'rmi
   ://michigan:54490/EboCacheCoordinator/exteNdDirectorCacheCoordinator'.
  EboCacheLog|5|11/11/03|15:20:09:424|<Register> this exteNd Director instance with the
    cache coordinator...
  EboCacheLog|5|11/11/03|15:20:09:464|<Connect OK> connected to the cache coordinator
     successfully.

A corresponding message is generated on the Cache Coordinator. The serverID in the example is clone1. The client count (which is 1 in the example) indicates the number of servers currently connected:

  EboCacheLog|5|11/11/03|15:20:09:464|<Register> a client [ServerID: <clone1:
    c373e9f8cc736a65b1c7444553544200> AppID: <c373e9f8cc46d5b9b9d4444553544200>].
  
  EboCacheLog|4|11/11/03|15:20:09:464|<Registered client count> for [AppID:
    <c373e9f8cc46d5b9b9d4444553544200>] is: 1.

Cache invalidation message logging

The first example shows output triggered on the server instance when putObjectInCache() is called (with state=EboState.UPDATE) from an EbiSrvLifetimeCacheHolder:

  EboCacheLog|5|11/11/03|15:27:01:416|<Send notification> from [ServerID: <clone1:
  c373e9f8cc736a65b1c7444553544200> AppID: <c373e9f8cc46d5b9b9d4444553544200>] 
  to cache coordinator, message: cached content with [key: <admin> cache holder ID: <
  DirectoryService.UserCacheHolder.exteNd Server>] has been modified [state 113].

Here is the corresponding output on the Cache Coordinator server console:

  EboCacheLog|5|11/11/03|15:27:01:426|<Receive notification> from 
  [ServerID: <clone1:c373e9f8cc736a65b1c7444553544200> AppID: <c373e9f8cc46d5b9b9d4444553544200>]
  message: cached content with [key: <admin> from cache holder: <DirectoryService.
  UserCacheHolder.exteNd Server>] has been modified [state: 113].

The second example shows output triggered when the removeObjectInCache() method is call from EbiSrvLifetimeCacheHolder:

  EboCacheLog|5|11/11/03|15:32:11:572|<Send notification> from [ServerID: <clone1:
  c373e9f8cc736a65b1c7444553544200> AppID: <c373e9f8cc46d5b9b9d4444553544200>] to
  cache coordinator, message: cached content with [key: <PacPortletDetails.html> c
  ache holder ID: <testCluster.PageCacheHolder>] has been removed [state 112].

Here is the corresponding output from the Cache Coordinator server console:

  EboCacheLog|5|11/11/03|15:32:11:592|<Receive notification> from 
  [ServerID: <clone1:c373e9f8cc736a65b1c7444553544200> AppID: <c373e9f8cc46d5b9b9d4444553544200>]
  message: cached content with [key: <PacPortletDetails.html> from cache holder: <
  testCluster.PageCacheHolder>] has been removed [state: 112].

Remote Cache Coordinator administration

You can get an EbiRemoteCoordinatorAdmin object for the Cache Manager by calling EbiCacheManager.getRemoteCoordinatorAdmin(). This object allows you to administrator the Cache Coordinator from any application server instance. For example, you can get the list of registered server instances that are currently connecting to the Cache Coordinator. You might implement a servlet or a portlet to perform remote administration

For more information, see EbiRemoteCoordinatorAdmin in API Reference.