2.2 Index Tab

Use the Index tab to manage the indexing engine, the kinds and sizes of attachments that are indexed, index optimization, and so on.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab

The Index tab contains the following panels:

2.2.1 Indexing Engine Panel

Retain utilizes two index engine choices. These choices are set at install or upgrade. The index engine is the same for both the Standard High Performance Index and the High Availability Index options, (HAI). However, while the Standard engine is installed with the Retain Server, the HAI is a cluster of index engines, managers, and replicas which provide scalability and disaster protection. The standard index engine will perform better than the HAI under all but the heaviest loads. (Index engine speed is limited by disk IO for the Standard indexer, and by Disk IO and network speed for the HAI.)

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Indexing Engine Panel

Table 2-5 Using the Indexing Engine Panel

Option, Field, or Sub-panel

Information and/or Action

Connect to this Index Manager:

Index Manager Hostname:

The DNS name of the server where the index manager is installed.

Index Manager Port:

The port on which the index manager communicates. (Default is 9081.)

Index Manager Ensemble:

Refresh Index Configuration button

Use this to refresh the configuration settings.

Index Schema:

The version of the index schema

Index Cluster:

  • Indexing Node

  • Core

  • Shard

  • Cluster State

  • Connectivity

Information about the index cluster

  • IP address of the node

  • Name of the core

  • Shard number

  • Role of the node

  • Status of the connection

Most Recent Index Maintenance:

Last Backup of Indexes

Date and time of last index backup

Last Index Optimization

Date and time of last index optimization

Optimize Index Now button

Click this to optimize the index immediately.

Index Maintenance and Optimization

You can also configure how often indexes get backed up and how often the indexes are optimized for speed.

Use the Optimize Index Now button if you need to optimize the index immediately rather than waiting for the scheduled cycle. While the optimization proceeds, the Last Index Optimization field changes to In Process. When the process finishes, the date and time display, and the indexer log is updated.

WARNING:Index optimization will temporarily double the amount of index storage used as new index files are written. If a job is running the space use can triple. It is recommended to run index optimization on the weekend while no jobs are running.

2.2.2 Indexing Panel

You can control what Retain indexes here by adding as many items as you wish to the list of attachment types to index. Items are listed (in order) by type, extension, archived form (extractor used), and maximum stream size and file size. Choose whether to index the attachment based on its filename extension or its MIME type (the content itself). You also choose which extractor to use to index the attachment. Select as many as you need. If an attachment type is common in the system and needs to be indexed, but is not already indexed, configure the row above the list with the information, then click Add.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Indexing Panel

Table 2-6 Using the Indexing Panel

Option, Field, or Sub-panel

Information and/or Action

Extension/MIME drop-down

Use this to specify whether the item is identified by a filename extension or by its MIME type.

Item:

The file extention or application designation

Action button

Use these to add a new item or delete an existing one.

Extension/MIME list

A list of various applications, beginning with exalead.

2.2.3 Rebuilding Indexes Panel

Rarely, usually due to hardware failure, Retain’s search indexes can become incomplete or even damaged. Acting under direction from Retain support, you can use this panel to return the index to health

WARNING:Rebuilding indexes consumes significant amounts of CPU, RAM, disk IO and time.

Back up the Retain database before proceeding. Then use this panel only under the direction of a Retain customer support engineer.

Path: Retain Server Manager > Configuration > Server Configuration > Index Tab > Rebuilding Indexes Panel

Table 2-7 Using the Rebuilding Indexes Panel

Option, Field, or Sub-panel

Information and/or Action

Reindex Failed Messages button

Click this when one or more parts of the message failed to index. Existing indexes are updated with changes. Searches will work as normal during the reindex process.

Reindex All Messages button

Click this to reindex all of the messages in the system and update the existing index as required. Searches continue to work.

Delete Index button

Click this to delete the existing index and create a new one by indexing all messages in the system. Initially, no results are returned, but searches become more accurate over time as reindexing continues.

 

 

NOTE:Depending on the number of messages to be reindexed, this process could take a long time. Most systems index 2M - 8M messages per day, which is dependent upon various hardware factors.

There are 3 options:

  1. Reindex Failed Messages: reindexes messages where one or more parts of the message failed to index. Existing indexes are updated with changes. Searches will work as normal during the reindex process.

  2. Reindex All Messages: reindexes all messages and updates existing index with changes. Searches will work as normal during the reindex process.

  3. Delete Index: Deletes the current index and reindexes all messages. Searches will not be accurate until the reindex completes.

Index performance can be increased by the number of threads allowed to the indexer. Again this uses significant amounts of CPU, RAM and disk IO and any one of these can limit indexing and Web UI performance.

To increase indexing threads:

  1. Stop tomcat

  2. Edit ../RetainServer/WEB-INF/classes/config/solrcloud.indexing.properties

  3. At the end of the file, change the setting: indexingThreads=0 to indexingThreads=[1 - 24] (You should not go higher than 10 times the number of CPU cores; otherwise, the overhead of CPU slicing would actually work against you.)

  4. Start tomcat

You will also need to keep in account users needing to access their Retain mailboxes and perform searches. Increasing threads could slow down the web UI response, so you'll need to experiment with what works best for your system.