9.2 Index Tab

Use the Index tab to manage the indexing engine, the kinds and sizes of attachments that are indexed, index optimization, and so on.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab

The Index tab contains the following panels:

9.2.1 Indexing Engine Panel

Retain utilizes two index engine choices. These choices are set at install or upgrade. The index engine is the same between the Standard High Performance Index and the High Availability Index options, (HAI). However, while the Standard engine is nested with the Retain Server, the HAI is a cluster of index engines, managers, and replicas which provide scalability and disaster insurance. The standard index engine will perform better than the HAI under all but heaviest loads. (Index engine speed is limited by disk IO for the Standard, and Disk IO and network speed. The trade off of performance and bottleneck moved in the favor of HAI when the system is very large and under large demand.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Indexing Engine Panel

Table 9-5 Using the Indexing Engine Panel

Option, Field, or Sub-panel

Information and/or Action

Connect to this Index Manager:

Index Manager Hostname:

<description>

Index Manager Port:

<description>

Index Manager Ensemble:

Refresh Index Configuration button

<description>

Index Schema: 403

<description>

Index Cluster:

  • Indexing Node

  • Core

  • Shard

  • Cluster State

  • Connectivity

<description>

Indexing Node

<description>

Indexed: X , unindexed: X, failed: X

<description>

Most Recent Index Maintenance:

Last Backup of Indexes

<description>

Last Index Optimization

<description>

Optimize Index Now button

<description>

Index Maintenance and Optimization

You can also configure how often indexes get backed up and how often the indexes are optimized for speed.

Use the Optimize Index Now button if you need to optimize the index immediately rather than waiting for the scheduled cycle. While the optimization proceeds, the Last Index Optimization field changes to In Process. When the process finishes, the date and time display, and the indexer log is updated.

WARNING:Index optimization will temporarily double the amount of index storage used as new index files are written. If a job is running the space use can triple. It is recommended to run index optimization on the weekend while no jobs are running.

9.2.2 Indexing Panel

You can control what Retain indexes here. You may add as many items as you wish to the list of attachment types to index. Note the explanation at the top of the table. The items are listed (in order) by type, extension, archived form (extractor used), and maximum stream size and file size. Choose whether to index the attachment based on its filename extension or its MIME type (the content itself). You also choose which extractor to use to index the attachment. Select as many as you need. If an attachment type is common in the system and needs to be indexed, but is not already indexed, it may be added by using the ‘add’ row.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Indexing Panel

Table 9-6 Using the Indexing Panel

Option, Field, or Sub-panel

Information and/or Action

Extension/MIME drop-down

<description>

Item:

<description>

Extractor drop-down

<description>

Action button

<description>

Extension/MIME list

<description>

9.2.3 Rebuilding Indexes Panel

WARNING:Use this panel only under the direction of a Retain customer support engineer. You should first back up your Retain database.

Rebuilding indexes uses significant amounts of CPU, RAM, disk IO and time.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Rebuilding Indexes Panel

Table 9-7 Using the Rebuilding Indexes Panel

Option, Field, or Sub-panel

Information and/or Action

Re-Index Failed Messages button

<description>

Re-index All Messages button

<description>

Delete Index button

<description>

Under rare circumstances - and usually due to hardware failure/issues - the indexes used by Retain when the search feature is invoked become damaged or files become missing. Resetting the index state of messages in the Retain database causes the Retain indexer to re-index the affected messages.

NOTE:Depending on the number of messages to be re-indexed, this process could take a long time. Most systems index 2M - 8M messages per day, which is dependent upon various hardware factors.

There are 3 options:

  1. Re-index Failed Messages: Re-indexes messages where one or more parts of the message failed to index. Existing indexes are updated with changes. Searches will work as normal during the re-index process.

  2. Re-index All Messages: Re-indexes all messages and updates existing index with changes. Searches will work as normal during the re-index process.

  3. Delete Index: Deletes the current index and re-indexes all messages. Searches will not be accurate until the re-index completes.

Index performance can be increased by the number of threads allowed to the indexer. Again this uses significant amounts of CPU, RAM and disk IO and any one of these can limit indexing and Web UI performance.

To increase indexing threads:

  1. Stop tomcat

  2. Edit ../RetainServer/WEB-INF/classes/config/solrcloud.indexing.properties

  3. At the end of the file, change the setting: indexingThreads=0 to indexingThreads=[1 - 24] (You should not go higher than 10 times the number of CPU cores; otherwise, the overhead of CPU slicing would actually work against you.)

  4. Start tomcat

You will also need to keep in account users needing to access their Retain mailboxes and perform searches. Increasing threads could slow down the web UI response, so you'll need to experiment with what works best for your system.