This chapter describes how SilverStream uses server clustering to implement load balancing and failover and explains how to set up and maintain a clustered environment.
The chapter contains the following sections:
In order to accommodate high processing loads, SilverStream implements load balancing using server clustering.
What is a server cluster?
A server cluster is a set of servers running on different hosts that share the processing load. In SilverStream, a cluster is a group of independent systems working together as a single system connected to the same SilverMaster database catalog. In this configuration, a Web client interacts with a cluster as though that cluster were a single high-performance, highly reliable application server.
NOTE A cluster of servers can handle demand over time more efficiently than one large machine, because the sum of the bandwidth and resources of multiple machines is greater, and less expensive, than those of a single large machine.
Benefits of server clustering
SilverStream Server clustering provides the following benefits:
A SilverStream server cluster has five components, summarized in the following table.
Used by the Load Manager to redirect requests to SilverStream Servers | ||
The cluster components are configured as shown in the following figure.
Cluster requirements
A cluster configuration must meet these general requirements:
Cluster options
You have the following options within the cluster configuration:
NOTE When servers operate in a clustered environment, they might affect the way that scheduled and server-triggered business objects work. For more information, see the Programmer's Guide.
Each SilverStream Server has an intelligent caching mechanism for storing commonly accessed data in memory (such as HTML pages, forms, and security information). Reading this information from the database for each request would be very inefficient, so the server maintains this information in cache memory. Whenever the information is updated on the server (for example, a page or form has changed or new security permissions are applied), the server's cache is updated as well.
Maintaining this cache across multiple servers in a cluster is important for application and data consistency. In a clustered environment, multiple servers can simultaneously change the same data in the SilverStream system tables. This situation might leave data cached in memory in a corrupted or inconsistent state. The Cache Manager is responsible for preventing such conflicts by ensuring that servers in the cluster are notified when another server invalidates cache objects. Servers then discard their invalid cache entries and get an updated version of the resource the next time the object is needed.
The SilverStream Cache Manager can run on a separate machine or on any machine in the server cluster. The Cache Manager must be up and running before any servers in a cluster start. As part of server startup, the server reads the clustering information about the setup from the SilverMaster catalog and initiates contact with the Cache Manager. The Cache Manager registers the server's existence and identifies it as a member of the cluster.
If a server modifies an existing object, it notifies the Cache Manager that the specific object must be invalidated on other servers. Since all objects in a SilverStream environment can be denoted by a Universal Resource Identifier (URL), the server calls the Cache Manager to invalidate the specified URL for all servers. The SilverStream Cache Manager calls each registered server except the one that initiated the invalidation, and tells it to invalidate the object identified by the URL.
Example
A developer makes a change to a servlet in the development environment and publishes the new servlet to the cluster. The Cache Manager sees that the class has been updated in the database and instructs all SilverStream Servers in the cluster to invalidate their cache so that the new object can be read from the database and the updated copy retained in memory.
The Load Manager is a program that can run as a service or regular process on any machine in the same network as the cluster servers. The Load Manager's role is to keep track of each active server and its relative processing load (or weight) in the cluster. Based on this information, the Load Manager generates a distribution map, which it transmits to each Dispatcher in the configuration.
The SilverStream Load Manager must be up and running before any servers in a cluster start. As part of server startup, the server reads the clustering information about the setup from the SilverMaster catalog and initiates contact with the Load Manager to register the server's existence. The Load Manager first registers that the server is up. It then determines how to contact the server and factors the server into the distribution map. As part of Load Manager startup, the Load Manager reads the Dispatcher information about the setup from its property file and initiates contact with all Dispatchers so that it can transmit the distribution map in the future.
The distribution map, which is dynamically generated by the Load Manager, determines the processing load for each server in the cluster. The distribution weight for each server is represented by an integer between one and ten.
How the load is distributed
By default, a round-robin system is used, which means that each server in the cluster gets an equal number of hits over time and has the same weight (or no weight).
You can modify this distribution. For example, if you set Server 1 at 8, Server 2 at 4, and Server 3 at 2, over a period of time Server 1 would get twice as many hits as Server 2, and Server 2 would get twice as many hits as Server 3. The table that follows shows other examples of weight settings.
Server 1 is least hit, Server 2 eventually has twice as many hits, Server 3 eventually has three times the hits as Server 1. | ||
Server 3 eventually has 10 times as many hits as Server 1 and Server 2. |
For information on modifying the distribution, see Specifying a server's relative load weight.
The Dispatcher serves as the entry point for Web clients into a server cluster. That is, clients initially access the cluster through the Dispatcher's URL.
The Dispatcher is a lightweight HTTP/HTTPS-supported program that communicates with other cluster components to dispatch client requests to servers according to the distribution map that the Load Manager provides. The Dispatcher can be run as a service or as a regular process on any machine in the same network as the cluster servers. The Dispatcher and Load Manager can run on separate machines and on separate platforms, and do not need access to the SilverMaster database.
The first time the Dispatcher is called from the client, it finds the best server for the request, based on the distribution map that it receives from the Load Manager. Then, using a standard process called HTTP redirect, it redirects the request back to the client. The client then sends the request directly to the designated server.
Once a session has gone through a Dispatcher to a particular server, all client communication is then directly to the server, not through the Dispatcher; the redirection is not masked (that is, the user sees the URL for the server, not for the Dispatcher, in the browser).
The following figure shows the sequence in the HTTP redirect process.
The Dispatcher supports HTTPS (as well as both HTTP 1.0 and HTTP 1.1). This means you can install server certificates after you set up a cluster. See Administering a server cluster.
Using multiple Dispatchers
You can set up multiple Dispatchers in a cluster on separate machines. If one of the Dispatchers encounters a failure, users can manually redirect requests to other available Dispatchers (this does not happen automatically).
While the SilverStream Dispatcher is an excellent solution for many load balancing and failover needs, you might require functionality that is not available in the SilverStream Dispatcher.
You might want to use a third-party dispatching solution for the following reasons:
This capability is particularly useful in Internet applications where it is desirable for users to see only one host name (such as www.company.com), instead of a host name for each server and dispatcher.
For an example of a session-level failover implementation, see the Examples3_HTML sample database.
In situations like these, you might want to substitute a third-party dispatching solution for the SilverStream Load Manager and Dispatcher. You would still use the SilverStream Cache Manager to maintain a consistent cache and would still use the SMC to manage the server.
Any standard HTTP server load balancing solution should work with the SilverStream Server.
SilverStream provides system failover and recovery in the event of transient failure and persistent failure. If any component in a cluster fails, the failure is detected by a background program called SilverMonitor.
About SilverMonitor
SilverMonitor observes the state of the daemons, processes, and services running on the system. Each component of the server cluster is monitored by this program. If any component fails, SilverMonitor detects the failure and attempts to restart the component.
Usually, SilverMonitor ensures that failed components recover quickly. If the failure is persistent (due, for example, to a hardware failure) the program gives up after a predefined number of attempts, in order to conserve system resources.
NOTE SilverMonitor runs in each cluster by default. It is also provided as a server startup option through which you can define certain program parameters. For more information, see Using SilverMonitor.
If SilverMonitor cannot restart a server
When a server in the cluster goes down and is not restarted promptly, the following sequence occurs:
When the server is restarted
When the server is restarted, the following sequence occurs:
When a server fails, connected clients lose their connections. They need to either restart or send another request from the browser to the Dispatcher.
If restarting a server takes a significant amount of time, the cluster components force a reconfiguration of the cluster so that incoming client requests can be serviced as efficiently as possible. During a persistent failure, the active components respond as follows:
For information about installing SilverStream and configuring a database for use as a SilverMaster, see the Installation Guide.
The SilverStream clustering components (Cache Manager, Load Balancer, and Dispatcher) are installed with SilverStream Enterprise Edition, but not when installing the Developer Edition. The components can also be installed separately in the Enterprise Edition install by choosing Custom Install and selecting the Cluster Managers option.
For more information, see the Installation Guide.
For more information, see Starting the clustering components next.
If you installed the components as daemons or services, you can stop and restart them in this mode.
For more information, see Installing cluster servers.
For more information, see Creating the cluster profile.
For more information, see Restarting the clustered servers.
If the load balancing software is not currently running as a service on your network, you need to run the programs manually on the resident machine(s). If you are using the SilverStream Load Manager, execute the commands in the order shown in the procedure that follows.
To start the cluster components:
After you have installed a server that will contain the SilverMaster for the cluster, use the SilverStream Installation program to install the additional servers. When you install the additional servers, you will point each of them to the SilverMaster used by the first server (by definition, all servers in a cluster must use the same SilverMaster).
This section provides instructions for installation on Windows NT and UNIX. If you need more help with the installation, see the Installation Guide.
To install cluster servers (Windows NT):
In terms of software configuration, each machine in the cluster must be a mirror copy of the other. This means that:
After the software has been installed, you will be prompted to enter your license string for the server.
If this results in an error, you probably misentered information about the existing SilverMaster database. Go back and respecify that information.
After adding each server to the cluster, you can use the SMC to create the cluster profile, as described on Creating the cluster profile.
To install cluster servers (UNIX):
install.sh
) on the machine you want to add to the cluster.
NOTE Your ability to include a UNIX server in a cluster with NT servers may be affected by the availability of database drivers on both UNIX and NT.
In terms of software configuration, each machine in the cluster must be a mirror copy of the other. This means that:
NOTE If you have already added this server to your cluster using the SMC make sure that the port you choose here is the same as what you defined when adding the server to the cluster.
If this results in an error, you probably misentered information about the existing SilverMaster database. Go back and respecify that information.
After adding each server to the cluster, you can use the SMC to create the cluster profile, as described next.
Once you have the servers pointing to a single SilverMaster, you are ready to create the cluster profile using the SMC.
For more information, see Starting the clustering components.
The New Cluster wizard appears.
NOTE Be sure that the first server you add is the server containing the SilverMaster. All subsequent servers you add must be configured to use the same SilverMaster.
agserver.silverstream.com:50001
If you use port 80, you don't need to specify a port number. If SilverStream is not your primary Web server, you must change the port to a number above 5000. For instructions on changing ports, see Specifying general server properties.
The following dialog appears.
For information about changing the default ports, see Changing the clustering components' properties.
For information about changing the default ports, see Changing the clustering components' properties.
The following dialog appears.
The following describes the settings that appear in the dialog. When initially creating the cluster profile, you should specify the default ports, but you can later change the ports if necessary.
For information about changing the default ports, see Changing the clustering components' properties.
You return to the New Cluster dialog.
The new cluster profile is defined.
After defining the cluster profile, you must restart each server and, if you are running the SilverStream Load Manager, you must restart the Load Manager and Dispatcher(s).
For more information about restarting servers, see Restarting the SilverStream Server.
To restart the Load Manager and Dispatcher:
After creating a cluster profile, the SMC updates to display options specific to a clustered environment. The following example shows a cluster named Milton with two servers, ecraig and bhildebrand.
When you are working in a clustered environment, you need to be aware of three types of properties:
For a listing of these properties and information on setting them, see The httpd.props File.
For a listing of these properties, select a server in a cluster in the SMC and look at the properties listed in the panels. The listed properties include the server stored properties as well as the server local properties that are settable in the SMC. All server stored properties are settable in the SMC.
These are cluster-level properties: All servers in the cluster share the same values for the cluster shared properties. Most of the security properties are cluster shared properties.
For a listing of these properties, select a cluster in the SMC and look at the properties listed in the panels. All cluster shared properties are settable in the SMC.
How properties are set when a cluster is created or dissolved
When a cluster is created, a server that is included in the cluster carries over the values for the server local properties and server stored properties that were configured when the server was a standalone server. You can choose to retain these settings or change them at the server-level.
For information on setting server-level properties, see Setting server-level properties in a cluster.
However, a server included in a cluster does not carry over the values for the properties that are defined as cluster shared properties. You must reconfigure the cluster shared properties at the cluster level. The reason you have to do this is that servers that are included in the cluster might not have the same values for cluster shared properties when they were standalone. Therefore, when you create a new cluster profile, SilverStream sets all the cluster shared values to the default values. You can retain these settings or change them at the cluster level. Once you change a cluster-level property, the new value will be applied to all the servers in the cluster.
When a cluster is dissolved, all servers that used to be in the cluster are now standalone servers. The values of all properties that were configured while the servers were in the cluster are carried over to each standalone server.
For information on setting cluster-level properties, see the next section.
As mentioned above, when working in a clustered environment, some properties exist at the cluster level and some exist at the server level. If you select the cluster at the left-hand side of the SMC, you see the cluster-level properties. Most cluster properties are the same properties as for standalone servers and are documented elsewhere in this manual. The following section provides cross-references to the documentation for the cluster-level properties.
For information about the API for cluster-level properties, see com.sssw.rts.adminapi.AgiAdmCluster.
Configuration properties
The cluster-level Configuration properties are divided into the following SMC panels.
RMI/ORB properties for the cluster. For more information, see Specifying RMI settings. | |
Performance, Cache Manager, Load Balancer, and full text properties.
| |
Properties for handling e-mail. For more information, see Setting up mail on the server. | |
For more information, see Managing licenses. | |
Cache Manager and Load Manager properties. These are the same properties you specified when defining the cluster profile. You can edit these properties after creating a cluster. For information about these properties, see Creating the cluster profile. For information about changing these properties after you have created the profile, see Changing the clustering components' properties. | |
Settings for managing Enterprise JavaBeans. For more information, see Maintaining Enterprise JavaBeans. | |
Lets you add servers to the existing cluster profile, edit a server's properties (name and port number), and remove servers from the cluster profile. |
Security properties
The cluster-level Security properties are the same as the properties for standalone servers, except that DSA and RSA port information in clusters is at the server level.
For more information about the Security properties, see Security options.
Monitor properties
The cluster-level Monitor properties are the same as the properties for standalone servers.
For more information about the Monitor properties, see Monitoring server activity.
A server in a cluster has different properties than a standalone server, because many properties of a standalone server become cluster-level properties for servers in a cluster.
If you select a server in a cluster at the left-hand side of the SMC, you see the properties for a server in a cluster. Most properties are the same properties as for standalone servers and are documented elsewhere in this manual. The following section provides cross-references to the documentation for the properties of servers in clusters.
Configuration properties
The Configuration properties of a server in a cluster are divided into the following SMC panels.
General settings for the server. Every property except Server Relative Load Weight also exists for standalone servers. For more information, about Server Relative Load Weight, see Specifying a server's relative load weight. For more information about the other general properties, see Running the Server. | |
Debug, Performance, Cache, Cache Manager, and Load Balancer properties.
| |
Information about the databases in the cluster (that is, databases known to the cluster's SilverMaster). You can modify the minimum and maximum number of database connections for each server in the cluster. For more information, see Setting the maximum and minimum number of database connections. | |
For more information, see Managing client connections. |
Security properties
The Security properties for a server in a cluster concern DSA and RSA ports and cipher suites.
For more information about these Security properties, see Enabling RSA/DSA ports.
Monitor properties
The Monitor properties in a cluster are the same as the properties for standalone servers.
For more information about the Monitor properties, see Monitoring server activity.
You can specify the relative processing weight for each server in a cluster. The Load Manager uses that information to generate a distribution map that determines the processing load for each server at runtime.
For a description of how weighting works, see Distribution mapping.
To specify a server's relative load weight:
The SMC provides access to properties that control how the Cache Manager and Load Manager respond in the event of a system failure. Normally, you do not need to edit these properties.
The Cache Manager and Load Manager properties exist at both the cluster and server level. You can set the properties at the cluster level, which sets the values for each of the servers in the cluster. You can then override the values for any of the server in the cluster, but note that if you later change any of the properties at the cluster level, it will override settings you made at the server level.
The Cache Manager properties determine how the Cache Manager will respond when its connection with a server fails.
To set Cache Manager properties:
The following table describes each of the properties.
The Load Manager properties determine how the Load Manager will respond if its communication with a server fails.
To set Load Manager properties:
The following table describes each of the properties.
The number of seconds to delay between each connection retry in a series. | |
The number of times for the Load Balancer to begin a series of connection attempts with the server. | |
You can dissolve a cluster, which does the following:
You are asked to confirm your action.
You can change which hosts are running the clustering components (that is, the Load Manager, Dispatchers, and Cache Manager) as well as the ports that the components use.
Changing hosts
After creating a cluster profile, you might decide to run the clustering components on different hosts.
Changing ports
By default, all clustering components use port 54891 for their RMI port. In addition, the Dispatcher uses ports 54892, 54893, and 54894 for its HTTP, RSA, and DSA ports respectively. Typically, you won't change these port values.
But if you need to, you can change the ports after you create the cluster profile.
NOTE All clustering components must use the same RMI port.
NOTE If you changed the port for the Cache Manager, you must start it with the following command line:
SilverCacheMgr -p portNumber
For a SilverStream cluster to listen and serve on the HTTPS port, you must install a server certificate in all SilverStream servers in the cluster. In addition, if you are using the SilverStream dispatcher (SilverDispatcher), you need to install a certificate in each dispatcher in the cluster.
For more information about certificates and HTTPS/SSL see Using certificates.
About the server certificates
If you are using SilverDispatcher for the cluster, each SilverStream server must have a server certificate with the DNS name matching the server's host name (and, as mentioned, the SilverDispatcher machine must have its own server certificate).
If you are using a third-party hardware dispatcher (such as Cisco LocalDirector) that does URL masking--that is, masks all URLs such that the browser hitting any of the servers in the cluster sees the dispatcher's host name--then every SilverStream server must have a server certificate with the DNS name matching the dispatcher's host name.
Example
Say you have a SilverStream cluster with
If you are using SilverDispatcher, then you need to create four server certificates: one for the SilverDispatcher machine with the DNS name www.myhost.com, and one for each of the servers, with DNS names server1.myhost.com, server2.myhost.com, and server3.myhost.com.
If you are using a third-party URL-masking dispatcher, then you need to create only one server certificate with the DNS name www.myhost.com and upload it to each of the servers.
How to do it
The following procedures describe exactly what you need to do.
To generate server certificates:
If you are using SilverDispatcher, generate a different certificate for each server and SilverDispatcher used in the cluster, as described above.
If you are using a third-party dispatcher, generate one certificate with the dispatcher's DNS name, as described above.
For more information, see Generating an RSA server certificate or Generating a DSA server certificate.
To install certificates if you are using SilverDispatcher:
For more information, see Installing the certificate.
The server is now listening on its HTTPS port (default: 443 for RSA certificates and 444 for DSA certificates).
For more information, see Starting the clustering components.
For more information, see Installing the certificate.
The Dispatcher is now listening on its HTTPS port (default: 54893 for RSA certificates and 54894 for DSA certificates).
To install certificates if you are using a third-party dispatcher:
For more information, see Installing the certificate.
http-server.com.sssw.srv.https.cert.hostname=DispatcherName
where DispatcherName is the DNS name of the dispatcher.
For more information about httpd.props, see The httpd.props File.
The server is now listening on its HTTPS port (default: 443 for RSA certificates and 444 for DSA certificates).
For SilverServer full-text search functionality to work in clustered environment, Fulcrum needs to be configured for the network. You must configure Fulcrum SearchServer so that one of the servers in the cluster acts as the Fulcrum server and all the others are clients. All full-text indexing and searches will be directed to the single SearchServer engine.
This section includes instructions for configuring the server and the client on Windows NT.
ftsvcadm.exe
in the bin directory of the Fulcrum install directory). The following dialog appears.
The following dialog appears.
The Fulcrum Server Service Setup dialog appears.
The following dialog appears.
The description for usage in the following table assumes that the install directory for Fulcrum is C:\fulcrum
and the install directory for SilverStream is C:\silverstream
.
After configuring the server, configure all other machines in the cluster as clients as follows.
The following screen appears.
Change the FTNPATH parameter for the SearchServer_3.0 data source.
For example, if the remote server name is myserver, and the connectorname is tcp and the port parameter for tcp is 2048, you would specify FTNPATH as follows:
myserver/!tcp2048
The following table describes the syntax for this parameter.
The remote server where SearchServer was configured as a server (see Configuring the server). | |
Tell the server to automatically close any idle connections. | |
The network connector name (case-sensitive). This name should match the Client connector parameter defined in the server configuration described in Configuring the server. | |
NOTE SilverStream uses only the System SearchServer_3.0 data source. The Fulcrum installation program creates SearchServer_3.0 and Searchserver_3.7 data source entries in both the User and system DSN panels in the ODBC Datasource Administrator. To avoid confusion, you can delete the SearchServer_3.0 DSN from the User DSN panel and the SearchServer_3.7 DSN from both the User and the System DSN panels.
This section includes instructions for configuring the server and client on UNIX.
To configure the server on UNIX:
/etc/services
file to define fultext service and the port number for all Full Text Search Clients. The port number must be greater than or equal to 2048. For example:
fultext 2048/tcp # Fulcrum Server
inetd.conf file
for Fulcrum SearchServer program (ftserver) to be invoked automatically. For example (the following is a single line entry):
fultext stream tcp nowait root /home/silverstream/bin/agftserver agftserver -uSomeUser
inetd
process. Typically this is done with the following command:
kill -HUP process-id-of-inetdNOTE To find the process id of inetd you can run the following command:
ps -ef | grep inetd
To configure the client on UNIX:
.agprofile
file in the SilverStream install directory and add the following line to it.
FTNPATH=myserver/tcp2048 ; export FTNPATH
For information about FTNPATH, see Configuring the client.