4.5 Upgrading Retain 3.5.1.1 to Retain 4.7

IMPORTANT:If you need information and instructions for upgrading from a Retain version earlier than 3.5.1.1, see the Retain 4.7 Online Documentation.

Once Retain is updated to 3.5.1.1, you can start the Retain 4.7 upgrade process.

4.5.1 Upgrade changes

Retain 4 contains significant improvements over previous versions. The User Interface in Retain 4 is changed and a completely new search has been implemented. The new search contains a new, high-powered indexer and back end that allows searching in multiple new ways on each item to maximize relevant and intuitive results. Due to the increased ability, some system requirements have been increased.

4.5.2 Requirements

  • Storage Space: Best practice is to have available free space at least equal to 4 times the size of the current 3.5.1.1 index.

    IMPORTANT:The updater checks for this minimum amount and displays a warning if the free space is lower. However, it doesn’t prevent you from proceeding at your own risk.

    You can remove the old index after the update is complete. See Removing old indexes after Retain 4.7 has completed re-indexing.

    For more information about estimating storage requirements, see Planning Retain Storage Requirements.

  • Time Required: This depends on how large the Retain archive is. The upgrade includes two things:

    Updating the configuration database at first start-up and should finish in less than ten minutes.

    Migration to a new index, which can take significant time. However, the existing index and system are available and accessible during the entire process.

  • System Requirements Changes: The are increased to 12 GB RAM minimum. For more information, see System Requirements.

  • Licenses: Retain 4 requires new licenses. The upgraded system doesn’t function until the new licenses are applied.

    Make sure to run the install on every component in the installed Retain system so that all components are upgraded together. Any Workers and Servers not hosted on the same machine must have the installer run to be upgraded.

4.5.3 If Your System Began at Version 3.0 or Earlier

If your initial Retain system was version 3.0 or earlier, consider the following:

Item

Path

Action to Consider

Server Logging Level

Configuration > Server Configuration > Logging

  1. Set to Diagnostic, unless your root partition disk space is limited.

Worker Logging Level

Data Collection > Workers > Worker object > Logging

  1. Set to Diagnostic, unless your root partition disk space is limited.

Indexing

 

  1. Be aware that the default is now to index all content.

    You can adjust this if desired.

Worker Performance

Data Collection > Workers > Worker object > Connection tab

  1. Make sure the port is set to 48080. Older systems defaulted to 80, which routes through the web server and slows the system down.

  2. Avoid SSL, which cuts performance in half, unless your security policies require it.

Archiving from GroupWise Disaster Recovery

 

  1. You should always archive directly from the live post office and not from GroupWise Disaster Recovery.

    The only exception is as a temporary solution for cases of instability in the production post office.

GroupWise Module

 

  1. If you are archiving with the GroupWise Module, determine which storage flag Retain is currently using.

  2. During the upgrade, choose that same flag so that Retain can update the Item Store Flag in the Retain database for each user.

    If you choose incorrectly, you can change the flag in the module later. The next job performs a full dredge.

    NOTE:In 4.2 and later, the default for Scope in Profiles is New items if Date Range to Scan was set to All Messages and Duplicate Check was not set to Try to publish all messages.

    Also, the controls for the GroupWise Storage Flags have been moved from the Profile to the Module Configuration page.

Reporting & Monitoring Server

 

  1. If you use this tool, select the Core Settings > Disable the disk usage statistic option. This feature drains system resources.

  2. Make sure that the port under the Connection tab is set to 48080.

Consider the information in the following sections as they apply to your Retain deployment.

Profiles

GroupWise

Scope

One of the main points of having Retain is to have a complete archive of messages in your system because of data retention policies under which your organization must operate. If your email system's retention functionality is not understood or properly configured, users could purge items from their trash folder before Retain gets a chance to archive it. See the KnowledgeBase article: " How Retention Services and Item Store Flags Work".

Message Placement

Also, users make take a few days to organize their mail into folders. When the Retain archive job runs, it takes a "snapshot" of the user's mailbox as it exists at the time of archiving that mailbox.

If an Item A is in the main mailbox folder at the time of the job, it gets placed into the main mailbox folder in the Retain mailbox. If the user subsequently moves the item to another folder, Retain doesn’t remove the item from the main Retain mailbox folder; instead, it creates an additional pointer to the item in the folder to which the user moved the message. This means that future searches find the item in the main mailbox folder and in the folder where it was moved.

To avoid this, some customers set the Date Range to Scan to Number of days from job start (older) and set the number of days to a value between 7 - 21. This gives the users that extra time to organize their new email before Retain archives it.

Miscellaneous

To save disk space, enable the option Don't store MIME.822 attachments and then activating the settings:

  • Store/index Internet Headers

  • Include Routing Properties

These settings cause the data that is in the MIME.822 attachments and to be stored and indexed.

We also recommend only including shared folders that are owned by the mailbox. The other setting can cause errors in the job due to circular folder references.

Under Disabled/Expired Users, set "When Retain encounters a disabled or expired mailbox" setting to "Ignore".

Exchange/O365

Scope

One of the main points of having Retain is to have a complete archive of messages in your system because of data retention policies under which your organization must operate. If your email system's retention functionality is not understood or properly configured, users could purge items from their trash folder before Retain gets a chance to archive it. See the KnowledgeBase article: " How Retention Services and Item Store Flags Work".

Message Placement

Also, users make take a few days to organize their mail into folders. When the Retain archive job runs, it takes a "snapshot" of the user's mailbox as it exists at the time of archiving that mailbox.

If an Item A is in the main mailbox folder at the time of the job, it gets placed into the main mailbox folder in the Retain mailbox. If the user subsequently moves the item to another folder, Retain doesn’t remove the item from the main Retain mailbox folder; instead, it creates an additional pointer to the item in the folder to which the user moved the message. This means that future searches find the item in the main mailbox folder and in the folder where it was moved.

To avoid this, some customers set the Date Range to Scan to Number of days from job start (older) and set the number of days to a value between 7 - 21. This gives the users that extra time to organize their new email before Retain archives it.

Miscellaneous

Enable these settings:

  • Store/index Internet Headers

  • Include user's recoverable items.

We strongly recommend abandoning the practice of using journal mailboxes to meet retention requirements (ensuring nothing gets deleted before being archived). Journaling doubles the traffic on your Exchange server and can adversely impact performance. Instead, have Retain archive the Recoverable Items folder in each mailbox and configure Rolling In-Place Holds. See the Knowledge Base article: " How to Transition from Journaling to Rolling In-Place Hold for Exchange Archiving" on how to configure this. It also provides links to articles on other recommended settings in Exchange.

Reporting & Monitoring Server

This tool provides archive stats as well as archive job stats. In Retain 4, it becomes a very critical tool for identifying errors in your jobs because it not only shows the error counts, but through drill downs on the error numbers, an administrator can see the exact message on which the error occurred, the folder in which it is stored, its delivered date, and the error description.

If you use this tool, make sure that when configuring it, you check the box under Core Settings that reads, Disable the disk usage statistic. This utility drains the system and is scheduled for a redesign in the future. In the meantime, disable it. In Retain 4, this utility is disabled by default.

And, like the Worker, the port settings under the Connection tab should all be set to 48080 if possible so that the Server and the R&M Server are talking directly to one another's tomcat port.

4.5.4 Upgrade Process

  1. Log in to the Micro Focus Customer center and download Retain 4.7.

  2. Extract the archive into a new folder.

    On Linux, in the GUI right-click and select "Extract Here" or at a terminal prompt, "unzip Retain<version>.zip"

    On Windows, right-click and select "Extract All..."

  3. If on Linux, open a terminal window in the Retain install folder and make the bash script file executable by running the command: "chmod +x *.sh"

  4. Run the installer:

    On Linux: "./RetainInstall.sh"

    On Windows: "RetainInstall.exe"

  5. The installer checks the tomcat version that is installed and upgrades it to Tomcat 8.0

  6. Retain detects whether a previous version of Retain exists on the server. It reports what components have been installed and asks whether you want to only upgrade the current installation or add new components as well.

  7. If you choose to install additional components, you are prompted to select them.

  8. Then the installer asks whether you want to Upgrade Retain, preserving the existing configuration or Overwrite Retain and replace the configuration file but not erase data. This is useful in certain troubleshooting circumstances.

  9. After choosing to upgrade or overwrite Retain, the installation begins. Wait until the installation is finished.

4.5.5 Post-Install Tasks

  1. Browse to Retain Web Console.

  2. The upgrade may require a configuration database update, depending on the version of Retain you upgraded from. Select the ‘Enter Credentials’ button and enter the administrator account username and password. Select the button to start the migration.

    1. During the database Migration, do not interrupt the process. Wait until the process completes.

    2. Once the migration is complete, the system instructs you to restart tomcat.

    3. Once the update has completed, restart tomcat to initiate the new database.

    4. The initial tomcat restart takes a little bit longer due to database initialization and interface setup.

    5. After a few minutes, browse again to the Retain Management interface.

  3. If you use the GroupWise module, GroupWise timestamp usage by Retain has changed.

    1. You receive a warning page describing the change what you need to do.

    2. Enter the admin credentials.

    3. Choose which timestamp flag Retain should use in the Module.

    4. The Retain updates begins, and the interface is not available until the process completes. In Retain 4.2 and later, this process is done in the background, so the interface is available immediately, however jobs are disabled until the process is complete.

    5. When the migration is complete, restart tomcat. In Retain 4.2 and later, no restart of tomcat is needed, jobs begin again on schedule when the process completes.

  4. Login to the Retain Management interface. New licenses for Retain 4 must be immediately applied. These licenses should be provided by the sales representative. (Removing old licenses is not required, but may help to keep interface looking clean.)

    1. To install the new licenses, select the ‘Browse’ button from the ‘Submit License’ section at the bottom of the screen. Select the desired license and when back in the Retain interface, select ‘Submit License’ to upload the new license to the system. Repeat the process for each license required for the active and desired modules in the system.

    2. It is a good practice to remove expired licenses from the Retain system to keep the interface clean and to easily see any licensing issue. However, removing expired licenses is not required.

  5. Do the following:

    1. Immediately go to the Server Configuration page and select the Index tab.

    2. Select the Migrate to 4.0 indexer button.

    3. There are two index options: High Availability Indexer and the standard index. Both options utilize the same high performance index engine and provide new searching functionality; however, the 'standard' engine is embedded while the 'High Availability Indexer' is a scalable, external, cluster system for large systems. The default is set to the standard index. The standard index engine is very powerful and sufficient for everything up to extremely large or busy systems.

      If you wish to use the High Availability Index engine, a separate license is required. Please contact support for guidance, system planning, and installation.

    4. Once the migrate button has been pressed, the new indexer section is displayed.

    5. The Standard engine only requires the Retain admin credentials. Enter the administrator credentials then save the changes by selecting the save-changes button to continue.

    6. The index migration begins automatically. During the migration of the index, the active index is displayed at the top of the page. The active index displayed at the top of the page is still be available and fully functional. New search options and features are not available until the new index becomes active.

    7. The index page is updated every 10 seconds to display the progress. With no archive jobs, migrations of the index average 60 messages per second on a Linux server, and about 45 messages per second on Windows Server. Migration speed is limited by disk IO, (and network speed with external indexes). Large messages with large attachments take longer. If archival jobs are running, the speed of the migration depends on the size and activity of the archival job.

    8. When the index page updates showing the new index is running, the new index has automatically become active.

    9. Once the index migration is complete, the indexer must optimize the index files. This is done during the nightly maintenance. The larger or more segmented the indexes, the longer this requires. This may temporarily triple the disk space the indexes use as the files are consolidated. It is a recommended to go to the Server Configuration | Maintenance tab and set the Enable Index Optimization to Saturday weekly. The first optimization is the longest and may take many hours, after that it usually only takes minutes and can be done nightly.

  6. The upgrade is complete.

4.5.6 Removing old indexes after Retain 4.7 has completed re-indexing

All items are re-indexed by the new indexing engine. During the index migration the Retain 3 indexes are available as legacy search, which is automatically disabled once the migration process completes.

You can find the location of your indexes in your ASConfig file. In that file there be a line that reads your path: indexPath

  • Windows: C:\Program Files\Beginfinite\retain\RetainServer\WEB-INF\cfg\ASConfig.cfg

  • Linux: /opt/beginfinite/retain/RetainServer/WEB-INF/cfg/ASConfig.cfg

Be very careful with your index directory! You can delete all of the files except the solrhome directory, the zoo_data directory (do not delete this non-essential file while Retain is running.), and the log.1 file.

4.5.7 Tool Compatibility with Retain 3.x (Lucene) Indexes

Not all tools are compatible with the Retain 3.x (Lucene) indexes.

Tool

Compatibility

Retain web console

Compatible

Publisher

Not compatible

Outlook Plugin

Not compatible

Retain App for Android Phones

Not compatible

Retain App for iOS Phones

Not compatible

Retain Single Sign-on plugin for GroupWise Client

Does not use indexes

Retain Single Sign-on plugin for GroupWise WebAccess

Does not use indexes

GroupWise Archive Migrator/PAM Tool

Does not use indexes

PST Importer

Does not use indexes

PowerShell Sync Script

Does not use indexes