AppNote: Effective use of Advanced Trending Agent

Novell Cool Solutions: AppNote
By Alok Panda, Bijaya Kumar Singh

Digg This - Slashdot This

Posted: 14 Oct 2004

Bijaya Kumar Singh
Senior Software Engineer
SBijaya@novell.com

Alok Panda
Software Engineer
PAlok@novell.com

1. Introduction
2. Understanding Advanced Trending Agent: 2.1 Architecture of Advanced Trending Agent; 2.2 How the Advanced Trending Agent works
3. Using Advanced Trending Agent: 3.1 How to configure a MIB OID for Advanced Trending Agent; 3.2 Configuring the trend configuration file; 3.3 Setting the Trending Agent to trend on new parameters; 3.4 Checking if Control entries are created; 3.5 Additional considerations for Linux agents
4. Integration with View Builder: 4.1 How to create views using View Builder
5. How to get trend data for a MIB OID
6. Conclusion

1. Introduction

Novell ZENworks 6.5 Management and Monitoring Services (MMS) is a comprehensive Network Management Solution that provides SNMP-based network management and monitoring capabilities with a full-featured SNMP management console.

In a corporate network there are different types of servers which run various types of services and applications. There are different parameters for each of these servers that are critical for a system administrator to monitor continuously, with any variation from the standard behavior defined and reported.

Advanced Trending Agent (ATA) is an application that collects and stores the trend data for any SNMP-instrumented parameter on any of the managed devices. The collected data is exposed through an SNMP interface. This data can be used to provide long term and short term trend graphs of the configured parameters. It can also be configured with threshold values to generate a trap when the value exceeds or falls below the threshold value.

The AppNote is divided into four sections:

Understanding Advanced Trending Agent explains how the trending agent works and the implementation of various MIBs by Advanced Trending Agent.

Using the Advanced Trending Agent explains in detail how to configure different parameters on a managed server to be trended upon, and how to verify if the MIB OID's configured are monitored by the agent or not.

Integration with the View Builder explains how different views can be created by the Custom MIB View feature of ZENworks, using the parameters monitored by the Trending agent.

How to get trend data for MIB OID explains ways to get the trend data to be used by any third-party tool.

2. Understanding Advanced Trending Agent

2.1 Architecture of the Advanced Trending Agent

This section describes the architecture of the ATA from the user point of view.

The ATA implements the usr history group of the standard RMON-II(rfc 2021) and Novell's RMON-II extension MIB. For detailed help for understanding the usr history and RMON-II extension MIB, refer to the corresponding MIB definition. These are shipped as part of the default MIB pool in the MMS.

The following MIB tables are populated with trending related information:

MIB tables in the UsrHistory group (in RMON-II) are

usrHistoryControlTable
usrHistoryObjectTable
usrHistoryTable

MIB tables in the RMON-II extension group are

usrHistoryExtControlTable
usrHistoryExtObjectTable
usrHistoryExtTable

We are going to use "table name.entry name.column name" for referring to a MIB variable.

The usrHistoryTable. UsrHistoryEntry. usrHistoryAbsValue contains the actual sample data of the trended MIB OID and the usrHistoryExtTable.usrHistoryExtEntry. usrHistoryExtSampleTimeStamp contains the timestamp of the sample.

The following table specifies the list of configuration parameters populated in various MIB tables.

Configuration Parameter	MIB Table Name.entry name.column
MIBVariable	UsrHistoryObjectTable.usrHistoryObjectEntry.usrHistoryObjectVariable usrHistoryExtControlTable.usrHistoryExtControlEntry.usrHistoryExtControlObjectVariable
SampleType	usrHistoryObjectTable. usrHistoryObjectEntry .usrHistorySampleType
MIBVariable TrendingStatus	usrHistoryExtControlTable.usrHistoryExtControlEntry.usrHistoryExtControlTrendingStatus usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectTrendingStatus
SampleInterval	usrHistoryControlTable.usrHistoryControlEntry.usrHistoryControlInterval
Buckets	usrHistoryControlTable.usrHistoryControlEntry.usrHistoryControlBucketRequests
MIBVariable DisplayName	usrHistoryExtControlTable.usrHistoryExtControlEntry.usrHistoryExtControlDisplayName usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectIdentifier
MIBVariableType	usrHistoryExtControlTable.usrHistoryExtControlEntry.usrHistoryExtControlObjectType
IdentifierOID	usrHistoryExtControlTable.usrHistoryExtControlEntry.usrHistoryExtControlIdentifier
Threshold AlarmGenerationStatus	usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectThresholdStatus
AlarmType	usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectThresholdType
ThresholdRising	usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectRisingThreshold
ThresholdFalling	usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectFallingThreshold
AlarmSeverity	usrHistoryExtObjectTable.usrHistoryExtObjectEntry.usrHistoryExtObjectAlarmSeverity

Table -1. Configuration parameters

2.2 How the Trending Agent works

This section gives a high level view of the working of ATA.

When the trending agent is loaded, it tries to load the existing samples, which it has taken earlier from the disk. If it does not find any existing samples, it starts taking fresh samples based on the configuration provided.

When the trending agent loads, it samples all the MIB OIDs, according to the sampling interval for that entry.

For each sample taken, it compares the sample values with the Threshold parameter setting. Whenever the sampled value crosses the specified rising threshold, a Trap is generated. No further Traps are generated until the sampled value goes below the rising threshold and goes up again. A similar method is used for the falling threshold.

Since the ATA maintains a persistent sample, in a certain interval the sample data is saved to the disk.

3. Using Advanced Trending Agent

The ATA starts trending on MIB variables specified in a configuration file. On the managed devices where ATA is installed, the configuration file is located at the following path:

NetWare: installation_path/advtrend/advtrend.ini
Windows: installation_path/advtrend/ini/advtrend.ini
Linux: /etc/opt/novell/zenworks/zfs-mms-advtrend.conf

This file can be used to specify various trending parameters for ATA. The configuration file follows a standard INI file format. Each parameter starts with a new section in the configuration file. There are different keys in each section which define a particular attribute for the MIB OID to be trended. Each section in the configuration file and each key within a section are unique. See the Administration Guide for details about configuring a trend configuration file at http://www.novell.com/documentation/zenworks65/index.html

Let's consider a scenario where we want to monitor a Linux server. The parameters to monitor on the managed Linux server are CPU utilization, memory utilization, Disk read/ write and the network traffic to and from that machine. The following steps explain how to configure the ATA to trend on these parameters, how to create views to see the trend of these variables, and how alarms are generated when the threshold is crossed.

3.1 How to configure a MIB OID to be Trended by Advanced Trending Agent

MMS provides MIB for managing some of the parameters for a Linux server. This MIB can be found by searching for linuxServer using MIB browser search utility, which is a SNMP tool that comes with MMS. This will help in finding the OIDs for the variables that are to be configured for trending.

Note: The Linux MIB can be found at "iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).novell(23).mibDoc(2).linuxServer(101)".

3.2 Configuring the Trending INI file

Let's take the example of CPU utilization as the parameter to be configured for trending. This parameter is already present in the configuration file on a Linux server. We will use this example to explain how to configure a particular parameter. The same guidelines can be followed to configure any other parameter. There are alternate values for each key of a particular section; for details about alternate values refer to the Administration guide at http://www.novell.com/documentation/zenworks65/index.html

In the advanced trending agent configuration file, look at the section named [CPU Utilization].

MIBVariable is the MIB OID for that parameters. To find the MIB OID for CPU utilization, expand linuxServer MIB and select processorInfo->processorTable->processorEntry->pUtilization. This has a OID of 1.3.6.1.4.1.23.2.101.6.1.1.16. So MIBVariable=1.3.6.1.4.1.23.2.101.6.1.1.16

The type of this MIB variable is columnar. So MIBVariableType=2.

We want the trending agent to trend on this variable, so MIBVariabletrendingStatus=1.

CPU Utilization is the display name of this trend variable. This name can be used in the SNMP managers views. MIBVariableDisplayName=CPU Utilization

The sampling interval for the trending agent is 60 secs, so the SampleInterval=60.

The trending agent should collect the information for a period of 30 days -- so the number of buckets that is needed to keep the sampled values is 259200. Buckets=43200

Note: To calculate the number of buckets, convert the time period into seconds and divide it by the sampling interval. 30 days in secs = 30 * 24 * 60 * 60 = 2592000 seconds, the number of buckets necessary is = 2592000/60 = 43200. It is advisable to use small sample intervals for short-term trending and large sample intervals for long-term trending.

We want the trending agent to send a trap if the CPU Utilization exceeds 90%. This is a rising alarm. So AlarmType=1. If you want the trending agent to send a trap when the CPU Utilization falls below 10%, then the AlarmType=2.

ThresholdRising=90%
ThreshholdFalling=10%

The trending agent can send only one rising or falling trap at any point in time.

The severity of this alarm generated should be Major. Alarmseverity=2.

To generate traps when the threshold value is crossed, set the ThreshholdalarmGenerationStatus=1.

IdentifierOID
This key is used to uniquely identify a conceptual row in an SNMP table using one or more SNMP variables. For CPU utilization the identifier OID can be the ProcessorNum with OID 1.3.6.1.4.1.23.2.101.6.1.1.1.
IdentifierOID={ 1.3.6.1.4.1.23.2.101.6.1.1.1}

In the [Agent Config] section of the INI file, enter appropriate values for the trending agent.

ReadCommunity=public	#	The community name that the trending agent will use to get the values.
PurgeInterval=3600	#	The time interval in seconds, after which the agent will purge the trend data if it is not able to query the trended MIB variable.
MaxLogFileCount=1	#	The number of log files that will be created for Advanced Trending Agent
LogLevel=2	#	The debug level of the messages that will be logged in the log files.

After configuration of all the keys in this section, the section for CPU Utilization looks like the following:

[CPU Utilization]
MIBVariable=1.3.6.1.4.1.23.2.101.6.1.1.16
MIBVariableType=2
MIBVariableTrendingStatus=1
MIBVariableDisplayName=Processors
SampleInterval=60
SampleType=1
Buckets=43200
ThreshholdRising=90
ThreshholdFalling=10
AlarmType=1
AlarmSeverity=2
ThreshholdAlarmGenerationStatus=1
IdentifierOID={1.3.6.1.4.1.23.2.101.6.1.1.1}

Configure the other MIB OIDs, such as Memory utilization, disk reads, disk writes, interface In-octates, interface Out-octates in the ATA's configuration file.

Following is the snapshot of trending agent config file, after configuring some of these parameters:

Figure-1. Sample ATA configuration file

3.3 Setting the Trending Agent to trend on new parameters

After configuring all the parameters mentioned above, the advanced trending agent should be set to read the new parameters entered, so that it creates control entries and starts trending on those parameters. It can be done on the command prompt of the managed servers.

NetWare	:	advtrend reread
Linux & Windows	:	advtrend read_cfg

Note: On Windows the PATH variable should include the advtrend directory, where advtrend.exe is present and on Linux the PATH variable should include the bin directory of ATA.

If the user wants to reset all the trend data that is gathered and then start trending fresh on the configured variables, command line options are available.

advtrend reset

The reset option deletes all the previously gathered trend data and starts trending fresh.

3.4 Checking if Control entries are created

After the ATA completes reading the configuration file, it creates the control entries for those parameters to be trended upon. This can be verified with MIB Browser as well as the log files of the ATA. The log files are located in the installed directory for Windows and NetWare and /var/opt/novell/log/zenworks for Linux servers.

Through MIB Browser, browse to the following MIB table where the OIDs that are trended by ATA are listed. Check for the control entries in those MIB tables. For more details, read section 2 of this article.

The following figure (Figure-2) shows the table of control entries that are created by the ATA after it has read the configuration file in our example. Check for the MIB variable OID, that are entered in the ATA configuration file, whether they are present in the MIB table or not. If it is not present, then those control entries are not created and you need to verify the error related to that particular OID in the log file created by the ATA.

3.5 Additional considerations for Linux agents

Certain configurations are necessary on a Linux server for ATA to start trending. These are as mentioned below:

Configuring the community string for ATA
Configuring the trap destination
Providing access for ATA to the complete MIB tree

For details about configuration, please refer to the Administration Guide at http://www.novell.com/documentation/zenworks65/index.html.

4. Integration with View Builder

Customized views can be created using the variables that are trended by the ATA. The View Builder is available from the Tools menu. (Tools --> View Builder).

4.1 How to create views using View Builder

Follow these steps to create a view:

Invoke the view builder from Tools-->View Builder menu options.
Create a new view by clicking on the New button in the View Builder dialog box.
Enter a name for the view builder and add components to the view.
Add graph view and alarm view to the view builder and enter a name for the graph.
Select the OIDs that you have configured for trending agent from the MIB tree that is shown by the view builder and enter a Display name for the graph.
After adding all the OIDs and the MIB for which alarms will be shown, save the view.

Now this view can be invoked on any of the devices where the corresponding OID's are configured for trending. Figure-3 shows an example of a view where the SNMP In Packets have crossed the threshold and a corresponding alarm is shown.

Click image for larger view.

Figure-3. View with configured parameters

See the following article for details about creating custom views: http://www.novell.com/coolsolutions/zenworks/features/a_view_builder_zw.html.

5. How to get trend data for a MIB OID

This section explains how the various MIBs are populated for each trending parameter with a suitable example. This also explains the mechanism to extract trending information for any trended MIB OID.

This is particularly useful in the case when a third-party product needs trend information from the ATA for a trended SNMP parameter.

Let's consider an example where the configuration file is modified to trend on two MIB OIDs. One of them is a scalar and another is a columnar (i.e. part of the table) MIB OID. Refer to the next section for more information on various configuration parameters.

Incoming SNMP packets (1.3.6.1.2.1.11.1.0)
Incoming packets in a Interface (1.3.6.1.2.1.2.2.1.11)

Figure-4 Sample Configuration

In this section, for specific MIB OID, an explanation about extracting the trend data from the various MIB tools is given. The MIB tables are generated from the configuration in figure-4.

Taking the sample configuration in figure-4, let's see how various MIB tables are populated with various trending-related information.

In the above example, there is a scalar OID (SNMP In Packets) and another columnar OID (Interface In Packets). This particular managed device has two interfaces, which results in two instances for Interface In Packets. This in turn creates three rows (instances) in the usrHistoryControlTable, usrHistoryObjectTable, usrHistoryExtControlTable and usrHistoryExtObjectTable. Each instance corresponds to one control entry and one object entry. Refer figure-5 to figure-7 for these MIB tables.

Suppose we want to get the trend data of a columnar OID for "Interface In Packets". Let us assume that there are two interfaces on the managed device and we want to get the data on the first interface. We pick the first instance 1.3.6.1.2.1.2.2.1.11.1 (iftable.ifEntry.ifInUcastPkts.1) for trending.

Note: Refer to the MIB definitions carefully to find associations (index) between various MIB tables because this information is essential for retrieving data from various tables.

The following steps are required to extract the trend data and various configuration parameters for any specific trended MIB OID from different MIB tables.

We need to find out the location (index) of the required MIB OID in the usrHistoryObjectTable. If we do a SNMP Treewalk on usrHistoryObjectTable. usrHistoryObjectEntry. usrHistoryObjectVariable (i.e., 1.3.6.1.2.1.16.18.2.1.2) we get all the trended MIB OIDs (refer to figure-5). In this case, 1.3.6.1.2.1.16.18.2.1.2.2.1 has the MIB OID we are interested in. The index is 2.1.

Figure-5. Treewalk of UsrHistoryObjectVariable

If we follow the MIB definitions in the usrHistory group, we can extract the index of the usrHistoryControlTable directly from index 2.1 (from step-1). The associated index (row in table) in usrHistoryControlTable is 2, which contains some of the configuration information for the trended MIB OID. Refer to figure-6 for details.

Figure-6. UsrHistoryControlTable

The Novell proprietary usrHistoryExtControlTable & usrHistoryExtObjectTable are the extensions of the standard usrHistoryControlTable and usrHistoryExtObjectTable, respectively. Both of these MIB tables have the same index as that of usrHistoryControlTable and usrHistoryExtObjectTable. So we also have the indexes of usrHistoryExtControlTable & usrHistoryExtObjectTable are 2 and 2.1, respectively. In figure-7, the highlighted portion of the UsrHistoryExtObjectTable shows some of the configuration parameters with respect to index 2.1.

Figure-7. UsrHistoryExtObjectTable

Similarly, in figure-8, the highlighted portion of the UsrHistoryExtControlTable shows some of the configuration parameters of index 2.

Figure-8. UsrHistoryExtControlTable

The sample data for all the trended MIB OID is kept in the usrHistoryTable. If we do a SNMP Treewalk for usrHistoryTable.usrHistoryEntry.usrHistoryAbsValue.2 i.e., 1.3.6.1.2.1.16.18.3.1.4.2 (2 is the index of usrHistoryControlTable obtained in step-2), we get all the sample data for the trended MIB OID. Refer to figure-9 for the results of the above SNMP Treewalk.

Figure-9. Treewalk of usrHistoryTable.usrHistoryEntry.usrHistoryAbsValue.2

The time stamp when the sample was taken is stored in the usrHistoryExtTable. This also has the same index as that of usrHistoryTable. If we perform a SNMP Treewalk for usrHistoryExtTable.usrHistoryExtEntry.usrHistoryExtSampleTimeStamp.2 i.e., 1.3.6.1.4.1.23.2.102.1.3.1.2.2 (2 is the index of usrHistoryControlTable obtained in step-2), we get the corresponding time stamp for the sample taken in the step 4 from usrHistoryTable. Refer to figure-10 for the result of the above SNMP Treewalk.

Figure-10. Treewalk of usrHistoryExtTable.UsrHistoryExtEntry.UsrHistoryExtSampleTimeStamp.2

Using steps 1 to 3, we can retrieve all the configuration information for any specific trended MIB OID. Steps 4 and 5 gives us the trend data for any specific trended MIB OID.

6. Conclusion

In this AppNote, we have discussed how an Advanced Trending Agent works and how to use the Advanced Trending Agent effectively to monitor any SNMP OID on a managed server. It explains how to configure the trending agent and how to create views in ConsoleOne to see the trend and alarm information. It also explains ways for an administrator to retrieve the trend information and use it through any third-party tool.

Like what you see?
Sign up for our weekly newsletter.