Collector Development Guide
We've now covered the major development tasks involved in writing a Collector and making it work in a deployment environment. There are a few additional tasks that developers typically do to wrap the Collector up as a nice package, which is what we'll cover in this section.
Listing Supported Devices
The Collector infrastructure is based on the concept of supported devices/applications, meaning that each Collector declares support for a given set of products and versions. When importing Collectors into Event Source Management and deploying them, a list of supported products is presented first, and then a list of Collector(s) that support that product. Obviously, for this system to work the developer must declare which vendors, products, and versions they have written the Collector to support in terms of what data it will parse.
The file that controls this information is the
deviceSupport.xml file in the Collector directory. Simply edit the file and
add XML sections to describe the known list of products that your Collector supports. The existing template copy gives you an example which you
can extend as needed, here's the format:
- Containing element that holds the list of supported products
- Contains the specification for a single supported product/version - repeat this element for each version
- The vendor that sells/distributes this specific product, something like
- The product name, something like
Luminet. Ideally you should pull this from the vendor's website.
- The particular version or range of versions of the product that is supported. If you've tested your Collector with a few different
versions, you can specify something like
- A long-form description of this product (not currently used anywhere).
plugin.properties file contains properties related to the current release of the Collector, many of which are set based
on prompts at the time the plug-in is created. But you can edit this file to update some of these values as needed, but note that you should
not edit some of them. Here are the properties:
- This specifies a description of the plug-in; edit this to describe what the plug-in does and what it integrates with. This information is presented in the documentation and in the Event Source Manager UI.
- This property assigns a product category to all events (unless overridden in code) processed by this Collector - see the Observer section of the Taxonomy for details
- Although the SDK attempts to provide all the functionality you will need for most devices, we also support extensibility by embedding Java code directly in the Collector. This has been used on at least one occasion for a system for which there was no Connector at the time. If you need this functionality, you will need to add any Java JARs to this property so that they are in the classpath.
- The unique ID for this plug-in (autogenerated, DO NOT EDIT). When you create a new version of one of your Collectors, if for some reason you decide to "start fresh" and create a brand new Collector, but want the new one to replace the old when imported into ESM, then you can copy the UUID from the old Collector to the new one.
- This is used internally at NetIQ only.
Each Collector plug-in also includes a document template that you can use to document details about your Collector, how to set up event sources and
configure the Collector, provide data samples, that sort of thing. The template resides in the
doc subdirectory within the Collector and
can be edited with the free LibreOffice word processor. Note that the documentation template that you see only includes the sections that we expect
Collector developers to edit; that material is then merged with some boilerplate sections to generate a couple output PDF documents, one of which is embedded
directly in the Collector plug-in and one of which is external.
The documentation template and the expected documentation should be pretty self-explanatory once you open the document. There are some special features, however, which are common to all SDK documentation and are described here. After reviewing that material, note that Collectors in particular have some additional special replacement variables, including:
- This variable will be replaced by the name of the vendor of the primary product supported by this Collector (based on the directory structure).
- This variable will be replaced by the name of the primary product supported by this Collector (based on the directory structure).
- This variable will be replaced with the letter code for the observer type assigned to this Collector (see above, from
- This variable will be replaced with a formatted list of the products supported by this Collector, as read from the
- This variable will be replaced with the name of the default Connector this Collector uses.
- This variable will be replaced with the name of all the Connectors supported by this Collector.
- This variable will be replaced with a list of all the Connection Modes supported by this Collector.
The SDK leverages the power of Sentinel's Solution Packs to make it convenient to deliver supporting content — correlation rules, reports, maps, etc — along with a given Collector. When used for this purpose, we call the associated Pack a "Collector Pack".
Collector Packs actually have a bit more going on that standard Solution Packs, but you should review how Solution Packs work and how to develop them to start. Then, note these important points:
- Collector Packs are edited by right clicking on the Collector and selecting Edit Collector Pack.
- The template Collector Pack that is cloned into a newly-created Collector is empty; you have to add custom content for that Collector.
- However, there is a variety of template content that can automatically included in the Collector Pack (see below).
To explain a bit further, Collector Packs are a merge between locally-defined, custom content and content from a generic template Collector Pack. The template content will typically have a filter added to it when it is included in a Collector's Pack, so for example a generic "Authentication Grouped by Server" report in the template will become, say, a "SUSE Linux Authentication Grouped by Server" report when added to the SUSE Linux Collector.
It should be noted that this feature was primarily added to support earlier versions of Sentinel (Sentinel RD and early versions of Sentinel Log Manager) where modifying an existing report to add a filter for a specific Collector's data was not as easy as we wanted. With Sentinel Log Manager 1.2, however, and Sentinel 7, the Save as report feature now makes it extremely easy to take an existing report definition within the product and easily clone it to handle only SUSE Linux data, or Novell eDirectory, or event SUSE Linux only for the Engineering department. Given the greater flexibility of the newer method, we are moving away from providing per-Collector template content for the newer platforms.
This is still relevant for users of Sentinel RD and earlier versions of Sentinel Log Manager, however, so here's how you use it:
- Open the file
controls.excludeand review the entries.
- If any of the control categories sound like they could usefully be populated by data from your Collector (this is done entirely by taxonomy), remove the entry from the file.
- If you need more details on exactly what the template content looks like, for example to find out what the reports look like and what they filter on, you can review the Collector Base Solution Pack or the reports under the same category name in the SDK.
Collectors are versioned using a hybrid scheme that describes not only the version of that particular Collector but also the version of the SDK against which the Collector is built.
2011.1r4 Collector, for example, represents the fourth version of a particular Collector built against the 2011.1 SDK template.
Each time you release a new version of a Collector that you maintain, you should increment its release number to indicate to consumers that it is a new Collector, that they
should replace their old ones. Doing so is very simple: just edit the
release.properties file in the Collector directory and set the number to the new version. In fact,
a production build currently prompts you to increment this automatically; it assumes that if you are doing a final production build then maybe the next build should be
a new version.
This is the end of our Collector Development Guide, we hope you've found it useful. Feel free to explore further by looking at our additional development topics (listed below) or reading our API documentation. Thanks!
- Back up to Develop to Sentinel
Collector Development Guide
- Getting Started
- Initial Build
- Plug-in Contents
- Data Parsing
- Build Process
- Event Construction
- Connector Interaction
- Common Code
- Additional Information