6.2 Considerations for Storage Requirements

6.2.1 Operating System

The OS receives a number of updates over the life of a Retain server, so that should be provisioned for.

6.2.2 Retain Program

The Retain program is updated often as well. This is often placed on the same volume as the OS.

6.2.3 Archive Storage Area

The Archive contains the item bodies and attachments. This is generally the largest percentage of the storage requirements on a Retain server. The bodies and attachments are stored as BLOB (Binary Large OBject) files in a folder structure that starts with /00/00/00 and grows to /FF/FF/FF for 16.7 million directories. BLOB files are stored only once, Retain implements a single-instance storage system, only a single copy of identical items is stored. BLOB files never change, they are only created or removed.

The files are accessed when a message is opened in the Retain Search Message interface or downloaded by Retain Publisher.

6.2.4 Indexes

The indexes allow for fast search of the data in Retain. When searching for items in Retain the indexes are used to return the results.

For best search performance, the indexes should be placed on a fast disk and optimized regularly.

The Index requires periodic optimization. This is set under Server Configuration | Maintenance. The Indexer requires as much free disk space as currently used index space for optimization. It requires three times as much if optimizing during an archive job.

IMPORTANT:Prior to Retain 4.11, the Zookeeper runtime folder was embedded in the index folder under the Retain's base path. However, from Retain 4.11 onwards, as the zookeeper service supports Solr7 and Solr8, the Zookeeper folder is decoupled from the index folder and is placed inside Retain’s base path.

Solr stores configuration files in the runtime ZooKeeper directory to manage nodes, replica allocations and schemas.

With these changes in Retain 4.11, the Tomcat user should be the owner of the retain base directory, otherwise, the runtime creation of the Zookeeper directory within this base directory will fail which will prevent the Retain Server from starting.

To fetch the base path from ASConfig.cfg file, run the following command for Linux:

cat /opt/beginfinite/retain/RetainServer/WEB-INF/cfg/ASConfig.cfg |grep -i basepath 

<basePath>/home/retain</basePath>

To set the ownership of a base path to tomcat:www execute the following command:

chown tomcat:www /home/retain

6.2.5 Database

The database contains the header information for each item. For example: Sender, recipient, date received, and so on as well as pointers to the message body and attached files stored in the archive. Each item may be quite small but may be replicated many times in the database if there are many recipients.

When browsing messages, the items in the database is being viewed. On larger systems, the database is often placed on a dedicated database server.

6.2.6 Logs

The logs track the actions that Retain takes. They can become quite large but are compressed at the end of each day and removed after 10 days, by default. This can be changed under Server Configuration | Logging. Logs can be moved to another volume as described below.