6.1 Server Configuration

6.1.1 Server Configuration

The overall operation of Retain is configured in this section.

Each tab will be explained below.

Along the top, you will see configuration options for the Retain server and workers known to this server.

The most important topic is communications. These are the settings you have set when you initially installed Retain. You may make changes here.

After changing any communications options, it is strongly recommended that Tomcat be restarted immediately.

6.1.2 Communications tab

The settings for the Communications tab are set in the initial setup of Retain. These include the connection to the SQL Database server, the REST and Router connection information, and the notification or SMTP information. If any changes in the system or corrections are needed, they should be configured here.

6.1.3 Database Connectivity

The Database Connectivity tab contains the connection information for both the configuration and storage databases. This information should never change unless the database server is being migrated to a new location.

6.1.4 Notification

In this location, the admin user specifies what address receives notifications about the general system, as well as what SMTP Gateway to utilize to send these notifications. This is also used for forwarding messages from Retain to the SMTP Mail Server for transmission to the recipients.

6.1.5 Retain Server Connection

This location dictates the connection address, port, and protocol to which the Retain Server will listen for communications from the Router and from any other REST applications. The communication settings must be correct from the point of view of the applications trying to use it. The hostname or IP address must successfully connect to the Retain Server.

6.1.6 Index tab

The Index tab allows the management of the indexing engine as well as the ability to decide what kinds of attachments are indexed and what size.

Indexing Engine

Retain utilizes two index engine choices. These choices are set at install or upgrade. The index engine is the same between the Standard High Performance Index and the High Availability Index options, (HAI). However, while the Standard engine is nested with the Retain Server, the HAI is a cluster of index engines, managers, and replicas which provide scalability and disaster insurance. The standard index engine will perform better than the HAI under all but heaviest loads. (Index engine speed is limited by disk IO for the Standard, and Disk IO and network speed. The trade off of performance and bottleneck moved in the favor of HAI when the system is very large and under large demand.

Indexing

You can control what Retain indexes here. You may add as many items as you wish to the list of attachment types to index. Note the explanation at the top of the table. The items are listed (in order) by type, extension, archived form (extractor used), and maximum stream size and file size. Choose whether to index the attachment based on its filename extension or its MIME type (the content itself). You also choose which extractor to use to index the attachment. Select as many as you need. If an attachment type is common in the system and needs to be indexed, but is not already indexed, it may be added by using the ‘add’ row.

Rebuilding Indexes

Under rare circumstances - and usually due to hardware failure/issues - the indexes used by Retain when the search feature is invoked become damaged or files become missing. Resetting the index state of messages in the Retain database causes the Retain indexer to re-index the affected messages.

WARNING:This should only be done under the direction of a Retain customer support engineer. It is recommended that you first back up your Retain database. Rebuilding indexes will use significant amounts of CPU, RAM, disk IO and time.

NOTE:Depending on the number of messages to be re-indexed, this process could take a long time. Most systems index 2M - 8M messages per day, which is dependent upon various hardware factors.

There are 3 options:

  1. Re-index Failed Messages: Re-indexes messages where one or more parts of the message failed to index. Existing indexes are updated with changes. Searches will work as normal during the re-index process.

  2. Re-index All Messages: Re-indexes all messages and updates existing index with changes. Searches will work as normal during the re-index process.

  3. Delete Index: Deletes the current index and re-indexes all messages. Searches will not be accurate until the re-index completes.

Index performance can be increased by the number of threads allowed to the indexer. Again this will use significant amounts of CPU, RAM and disk IO and any one of these can limit indexing and Web UI performance.

To increase indexing threads:

  1. Stop tomcat

  2. Edit ../RetainServer/WEB-INF/classes/config/solrcloud.indexing.properties

  3. At the end of the file, change the setting: indexingThreads=0 to indexingThreads=[1 - 24] (You should not go higher than 10 times the number of CPU cores; otherwise, the overhead of CPU slicing would actually work against you.)

  4. Start tomcat

You will also need to keep in account users needing to access their Retain mailboxes and perform searches. Increasing threads could slow down the web UI response, so you'll need to experiment with what works best for your system.

6.1.7 Storage tab

The default storage path for your Retain archives is listed here. You can also manage encryption of the BLOB files in the archive.

Storage Panel

Retain uses a hybrid storage system. All message headers and metadata is stored in the database but all message text and attachments are stored as files on the file system in the location listed here (and subdirectories). This is done a little like GroupWise, where the OFFILES directory stores Binary Large Objects (BLOBs). All message bodies and attachments are stored in the archive on disk as BLOBs. In the case of Retain, this is transparent to the user.

Under advanced settings you may change the store path of attachments including message bodies, XML mappings, indexes, PDF exports, backs of the index files and the license files.

Encryption Panel

Files on disk may be encrypted. In version 4.5 Retain now supports AES-256 bit keys.

With Encryption set to “none”, jobs will run normally and the archived BLOBs will not be encrypted.

With Encryption set to “aes”, jobs will be suspended until a new key is generated. Once a key is generated archiving will resume and Retain will begin the process of encrypting the archived BLOBs. If there is a large amount of data in the archive this may take some time. Retain will encrypt the files during idle periods between jobs.

If you believe the key was compromised, you may revoke the key and generate a new key, and Retain will then decrypt and re-encrypt all the stored items with the new key, this may take some time.

To enable Encryption:

  1. Set the Encryption dropdown menu to AES.

  2. Click “Generate Key”, enter a password to have a new key generated. This password will be needed to revoke the key later. The password needs to be a minimum of 6 upper or lower case alphanumeric characters. Do not use special characters.

    NOTE:Do not lose this password!

  3. Retain will encrypt new items as they are stored and begin encrypting existing files with the new key in the background while the server is idle. This may take considerable time.

To Revoke a key:

  1. If you wish to revoke a key, click on “Revoke Key” to revoke the current key.

  2. Enter the key password. This is a security requirement so not anyone with admin Manager Server | Encryption Management rights can alter this setting.

  3. Jobs will be suspended until a new key is generated or Encryption is set to none.

6.1.8 Accounts tab

The accounts tab allows you to control the creation of accounts.

Account Management

Expire unused accounts after how many days: Enabling this will remove ANY account, including admin, not logged into for the set number of days (0=never expire)

Disable new accounts: will prevent new accounts from being enabled by default.

Prohibited logins: Block specific users from logging into Retain. Enter the username or email address and add or select and press Remove selected address.

Password strength

Open System vs. Closed System

Normally, Retain lets all mail system users log in. This is considered to be an “open” system. When that happens, Retain will check to see if a Retain account already exists and if not, it will create a new account for them and assign them to the group default.

Sometimes, you don’t want certain users to have access to the Retain archives. In this case, you may add these users to the list of Prohibited Logins. You do so by entering their name in the Address field and click “Add”.

To make a “closed” Retain system, simply click on “Disable New Accounts”. If you use this option, it means that you will have to manually create accounts in Retain for authorized users. In other words, the only people who can access your system will be people who you specifically create an account for.

In Retain, user accounts expire after 30 days of inactivity by default. You may choose the number of days or choose 0 for “accounts never expire”.

See “User Rights” for more information.

Password Strength

User-created passwords may be controlled for strength. By default, Retain accepts any password set by users. To require a higher security password, select the higher level desired. Requirements for the low, medium, and high settings are defined as:

Will accept any password

Low: Must be between 5 and 15 characters in length.

Medium: Must be between 5 and 20 characters in length, with at least 1 lower case characters, at least 1 upper case characters and at least 1 numerical characters.

High: Must be between 8 and 20 characters in length, with at least 2 lower case characters, at least 2 upper case characters, at least 2 numerical characters, and at least 2 special characters.. Also, the password will be checked against a dictionary.

KeyShield SingleSignOn

Retain supports the use of KeyShield SSO for users. To use the KeyShield client in coordination with Retain, Retain needs to have an open connection to the KeyShield server, the User ID alias, and the API key. Specify the KeyShield SSO Server URL, Alias, and API key. The Timeout is set in seconds, and may be anything required, 5 is recommended. Test the connection to ensure proper function.

When configured, Retain checks to see if the KeyShield client is running and if the user is currently logged in. If they are logged in, Retain checks the user against the specified KeyShield Server and then either fails authentication and sends users to the login page, or immediately passed them to their interface. The effect is that users who are already logged into the KeyShield client will not be required to login to Retain, but will be immediately taken to their appropriate interface.

Intruder Lockout

Accounts may be locked if multiple failed attempts are detected, according to the specified time window. This is useful to deny password cracking attempts on the server.

To enable Intruder Lockout, select the checkbox next to the ‘Enable Intruder Lockout’ option and save the changes. All changes will be immediate as soon as the save button is selected.

If a user has locked their account and requires immediate access to the system, all lockouts may be cleared. To clear any locked accounts, select the ‘clear lock outs’ button at the bottom of the page. There is no need to save changes; the clear command is immediate.

6.1.9 Maintenance tab

Maintaining a Retain system involves many tasks:

  • Backup Configuration (Embedded) Database

  • Compress Configuration Database

  • Indexing (All Indexes)

These items are all controlled here.

This configuration database – the “Embedded” database:

  • Should be backed up regularly

  • Contains your system address book

  • Also contains key information used by Retain

  • Configure how often it gets backed up here.

Index Maintenance

You can also configure how often indexes get backed up and how often the indexes are optimized for speed.

WARNING:Index optimization will temporarily double the amount of index storage used as new index files are written. If a job is running the space use can triple. It is recommended to run index optimization on the weekend while no jobs are running.

Retain also lists a history of backups and maintenance here.

NOTE:This maintenance section does not involve backup or maintenance of the data store. You must back that up yourself with your normal backup tools.

You must backup:

  • SQL Database

  • File System where your data is stored

6.1.10 Ignore Domains

Some mail systems allow for the addition of non-system, or external domains. Sometimes, you might do this to add external domains to the mail system address book. However, the mail system cannot pull up e-mail from these domains and you may want to tell Retain to ignore references to these external domains to avoid wasting time during data collection.

Specify these domains in this window.

6.1.11 Logging tab

Configure the system logging here. Normally, you want verbose logging. Diagnostic is ordinarily done just for troubleshooting purposes.

You also specify how long to keep logs here. By default, logs are deleted after 10 days. Logs may be compressed to save disk space.

6.1.12 Auditing Records

Retain now will create an auditing record of all actions, specified by the user, which are taken on a specific piece of mail. Auditing records can be removed automatically after a specified length of time.

The variable is set in days. All of the options associated with this feature are found under the Server Configuration page, Logging tab.

This logging option creates very detailed activity logs for the options selected. It is very important to know that if every option is turned on the logs can become extremely large. However, the audit log cannot be searched for any items or activity which is not configured to be logged. If activity on any of the offered items it must first be enabled here. It is highly recommended that an expiration date is set for the logs so that they are automatically removed from the system to avoid filling up your disk space. Selecting all options for logging will also adversely affect performance. Do not select all the options at one time unless requested by Support.

6.1.13 Miscellaneous tab

You may choose the default worker password here. One is automatically generated for you when you create a worker but you may change the password here if you like.

6.1.14 Export XML

This XML Export function is included in Retain in case you have an XML compliance mandate. You enable it here.

When selected, each attachment will have an XML export file of its parent message. In other words, an XML representation of the metadata is created and linked to the blobs as the messages are stored. There's nothing more done.

Ordinarily, you would not want to do this because it consumes enormous quantities of disk space, loads up your file system and degrades performance.

When you do NOT use the XML export function, you will benefit from Retain’s single-instance message storage and data compression to save disk space and improve performance.

6.1.15 Module Forwarding

Modules can be set to forward all new items to another location.

SMTP Forwarding

SMTP Forwarding is a feature which instructs Retain to forward a copy of any archived message data to the specified address and domain, when the message item is archived. For Module Forwarding to function, it must be configured here on the Server Configuration and also enabled on the specific desired module. Any module which has the SMTP Forward option enabled will send messages according to the settings configured here.

This setting is designed for exporting data to an external SMTP system for redundant archival. This is most often used for mobile, social, or blackberry message systems. SMTP Forwarding will take a text, pin, sms, or posted message and message data, convert it into a MIME file and send the message to the specified SMTP system. The Forward process checks for and sends any queued data once every 10 minutes. If a message is unable to be sent for any reason, after 5 days the MIME file will be saved to the local disk and may be reviewed and repaired by an administrator.

This feature is not recommended for use with any SMTP system which Retain is currently archiving. If Module Forwarding is enabled and configured to use the SMTP system which Retain is currently archiving, duplicate data will be archived; Mobile, Blackberry, and Social data will be doubled in the Retain archive.

DO NOT use SMTP Forwarding with any email module if Retain is archiving the destination SMTP Mail System. This will cause a feedback loop which will rapidly fill the archive and email system.

Requirements

  • Separate, working SMTP system

  • Mobile module requires the mobile app 4.0.1 or higher

Configuration

The SMTP Forwarding feature requires a SMTP Mail Server connection configured. Input the destination Mail Server's DNS name, protocol, and port.

The SMTP Mail From Address will be the address which displays as the 'from' address for the forwarded messages.

The SMTP To Address is the destination account for messages. If a single address is to be used, it should be a journaling mailbox. Retain can also send each message to a destination mailbox for each originating user. If it is desired to send the messages to each respective owner's mailbox, configure the 'SMTP To Address' with: {userid}@<yourdomain>.com

Retain will automatically use the userID of the originating device or account. Use of this feature requires that the SMTP server has an existing mail account for each user which matches the userID Retain shows for the mobile or Blackberry device.

If the destination SMTP server requires a login, provide an appropriate login. The login username and password will have no bearing on the 'from' address.

Press the “Test Connection” button to have Retain send a test message to the destination.

For troubleshooting, attempting to telnet from the Retain server to the SMTP server may provide useful information. Configuration data is stored in ASConfig.cfg.

FTP Forwarding

This feature instructs Retain to take a copy of any archived message data to the specified location, when the data item is archived. This feature must be configured here, in Server Configuration as well as selected on the desired system module configuration page.

FTP Forwarding is designed for exporting data to an external FTP server for redundant archival. While the FTP Forwarding service does not have the inherent danger of duplicating data as the SMTP Forwarding feature does, it is important to note that should the exported data be sent through any system the Retain system archives, it will duplicate data in the Retain archive. FTP forwarding simply sends a copy of the data in the format that it is received, (Mime, text, etc.). The FTP Forwarding service is run every 10 minutes, checking for any data queued for delivery.

Requirements

  • Separate, working FTP Server

  • Module and Server Configuration must both be configured

Configuration

The FTP Forwarding feature must have a configured FTP Server connection before the forward option will appear in the Module configuration pages. To configure the FTP Forwarding option fill out all the required information and then test the connection. Test results will be displayed after the test completes.

FTP Server is the connection address or DNS name of the FTP Server

FTP Security Protocol is the security of the receiving FTP server. Options are Unencrypted, SSL, or TLS.

FTP Port is the listening port of the FTP server.

FTP Upload Location is the path to the desired destination directory of the FTP Server. This must be the location as you would view it in an FTP client. FTP Username and Password are the login credentials to the FTP Server.

Once configured, select 'Test Connection' and then save the settings by selecting the disk icon at the top right of the page. Now the FTP Forwarding option has been configured the FTP option will appear in the module forwarding section of each individual module configured in the system. Only the modules which have had the FTP option saved will utilized FTP Forwarding. The feature must be configured in both places in order to function.

For troubleshooting, attempting to FTP from the Retain server to the FTP server may provide useful information. Configuration data is stored in ASConfig.cfg.

6.1.16 REST API

The REST API is an interface built into Retain which allows third-party applications to input data into the Retain archive. This API was developed to open new horizons to the Retain Archive for any application which conforms to the API requirements. Because of the access the API grants to applications, an additional API license file is required for each application. This license contains credentials and access rights. In order for a third-party application to utilize the REST API, the application must have a credentialed key provided by Micro Focus.

The REST API is an input protocol only. For Archive security, the API prohibits migrating data out of the Retain Archive. There are also limits placed in each applications API license file to protect the integrity of the data being input into the archive, based on the application.

Submit License

To gain a credentialed key for a third party application, contact a Micro Focus sales representative. Use of the REST API will require a REST API base license, and an application specific license for each desired application.

REST Collector

Some modules use the REST API for data collection. You need to create a collector to connect to the message system. You may have multiple REST collectors.

Create a REST Collector by providing:

  1. The Collector Name can be anything you want, but we recommend something easy to identify with the system it connects with. A Key and Secret will be created for the collector. The Key and Secret will be used on the message system side of the system for authentication.

  2. Then press "Add collector" to enable the collector.

  3. Export the collector allows you to export the Key and Secret to disk for use with the message system.

Delete removes the collector.