Novell Cool Solutions

ZLM Custom Reporting Utility ( v0.1)



By:

April 9, 2010 11:42 am

Reads:6,774

Comments:2

Score:Unrated

Print/PDF

License:
Freeware

Download zlm-reporting-utility

download url: /communities/media/zlm-reporting-utility.zip

Contents:

Introduction to ZLM Reporting

The existing reporting capabilities in ZENworks Linux Management allows you to generate:

  • Multi-columned custom reports with filtering properties which are customized to user specific needs and
  • Standard canned reports used directly to obtain the relevant logical information about objects like bundles, packages, policies inventory, logs, and devices.
  • The existing features of reporting allows administrators to collect standard statistical data to analyze or monitor the state or properties of the objects present at any point of time.

    Problem

    Currently, ZLM provides limited reporting capabilities related to hardware, bundles, devices, software packages, software updates and logs. The canned reports that are available are limited in scope and requires understanding of the zlm schema relations and mapping attributes to create customized reports as per the various requirements and usability.

    System Environment

    Server / Agent Requirement:

    The solution to generate different types of ZLM reports is applicable to ZENworks 7.2 Linux Management and ZENworks 7.3 Linux Management Servers installed with postgresql as an embedded or external database.
    The ZENworks 7.2 Linux Management agent or later must be registered to the server and must periodically update its data such as software and hardware inventory information to ensure that the reporting information is up-to-date and accurate.

    Solution Description

    This solution aims to provide details about the custom reporting utility which encompasses generating canned reports for over 25 different types for ZLM, and is based on the important requirements from customers who are using zlm since past few years. This ZLM reporting utility is an orchestration of different requirements, allowing you to quickly generate the HTML based reports about the information or content stored under ZLM schema (datastore and objectstore) across different correlated database tables, depending on the selected options. You can query the reports differently and generate reports on devices, bundles, packages, inventory, logs, and policies.

    The different types of custom or canned reports pertaining to different use cases are incorporated as part of this standalone utility. These reports are related to different types such as software updates for packages and bundles as applicable to devices, installed or failed reports for bundles or policies, bundles or packages installed on specific devices , measuring and comparing inventory packages, triggering updates re-computation within ZLM, and some other miscellaneous use cases which provides relevant information on ZLM managed entities to the end user.

    All the ZLM reports are categorized and explained below based on the reporting options available in the utility:

    • Package and Device Update Reports
    • Bundle Reports
    • Inventory Reports
    • Device Reports
    • General Reports and Utility

    Package and Device Update Reports :

    The updates are software packages which are of higher version than those installed on the devices.
    The updates available are shown against the devices as a box-shaped ( |X| ) icon.
    The updates may or may not be contained in any package bundles.
    The updates can be contained in either the assigned bundle version or the unassigned version of bundle.
    The updates can be either assigned to the device (Mandatory/High Priority) or can be unassigned (Optional / Low Priority).
    The updates are computed for devices when software inventory is reported from the managed devices on change of:

    • package / bundle on the device
    • bundle / bundle group assigned to device or device group

    The bundles within a catalog are excluded for updates unless the bundles are directly assigned to a device. The ZLM server preference compute-package-device-updates and agent ZMD setting software-inventory-enabled should be true to get the correct data reported and device should update its inventory as per the configured schedules.

    Note: The numbering in the below tables are based on the Reporting Option available in the utility:

    Option Reporting Title and Description
    1 Total (Assigned and Unassigned) Package Updates and its Count per Device :
    This report lists the imported packages in ZLM repository which are available as updates (mandatory/ optional) for each of the devices . It also provides a report to see the count of all updates packages for each such managed devices.
    2 Report Mandatory Package Updates (Assigned) and its Count per Device :
    This report provides all the packages which are available as mandatory or assigned updates for each of the devices. It also provides a report to see the count of such available update packages for the corresponding managed devices.
    3 Report Optional (Unassigned) Package Updates and its Count per Device :
    This report provides all the packages which are available as optional or unassigned updates for each of the devices. It also provides a report to see the count of available updates packages for the corresponding managed devices.
    4 Report Mandatory (Assigned) Package Update Bundles with its Version:
    This report provides the list of Package Bundles containing the Software Updates in any of its existing bundle versions where the bundle must be assigned to one or more managed devices either directly or as part of device group
    5 Report Mandatory (Assigned) Package Bundles with the updates packages per Device:

    The report provides the list of assigned package bundles containing Software Update for the existing devices. This report also provides the list of packages available as an update from these bundles and information on which managed devices these updates are available. The bundle version is not shown to retain the simplicity and readability of the reporting data.
    6 Report Optional (Unassigned) Package Bundles with the Updates Packages per Device:
    The report provides the list of unassigned package bundles containing Software Updates for the existing devices. This report also provides the list of packages available as an update from these bundles and the information on which managed devices these updates are available.
    10 Report All Patch Bundles with the Category and Available as Optional Updates per Device:
    This report lists all the Patch bundles containing the optional package updates. The patch bundle should have its name starting with “patch-” and should be of category type as optional, recommended or security. Such patch bundle updates are displayed for each of the devices which it is not currently assigned. The patch bundles include the YaST Online Update (YOU) Patches for SLES9 / OES 1 devices or ZYPP Patches mirrored from NU repository.

    Bundle Reports:

    The following reports are related to the Bundles ( Package, File, YOU and Dell Update Bundles). The devices must periodically update its bundles software inventory to the server to ensure that the data is consistent with the device where the bundle is deployed and installed. The timestamp related reports follow the UTC timezone as configured for the database. The zmd settings which control the installed bundles installed information as part of inventory are software-inventory-enabled(true), refresh-interval-software (86400) and real-time-package-updates(true)

    Option Reporting Title and Description
    7 Report all the Installed Bundles per Device :
    This report displays all the bundles installed on each of the managed devices. Some additional bundle and device properties are grouped and ordered in reports.
    8 Report count of Package Bundles installed per Device :
    This report shows the total count of Package bundles installed on the existing devices . The reports provides the required bundles and device properties.
    14 Report Bundles Installed for a given Device:
    This report lists installed bundles (guid, version, type) on a device whose alias name you provide as an input. It assumes that the software inventory has been updated to the device.
    24 Report the modification timestamps for all the versions of a given bundle :
    The report provides the details of timestamp when the given bundle was modified. The timestamp reported from the database is only in the standard UTC time zone which is the default database time zone and is not the local to timezone of the device.
    25 Report all the bundles created or modified in the last n days :
    This report lists all the bundles which were modified either manually or as part of regular mirroring in the last specified number of days. The timestamp that is considered here is database timestamp which follows the UTC timezone by default. The reported timestamp needs to be converted to local time zone based on the timezone offset of the location. This report will be useful in case the administrator wants to know if any bundles were modified by any user since the last given number of days or whether the bundle was modified as part of mirroring schedules. The mirroring schedules should be ideally less than the number of days.
    26 Report Patch Bundles (latest version) Updates Available per Device in the Last n Days :
    This report list all the patch bundles with the latest version which are available as updates ( mandatory / optional ) since the last n number of days. The patch bundles beginning with the name ‘patch-‘ and category as security, optional or recommended are considered. These patch bundles mirrored in ZLM are applicable to YaST Patch for SLE9 based platforms and ZYPP patches for SLE10/11 platforms.
    28 Report Listing the Installed or Failed Bundles per Device in the Last n Days Based on the Bundle’s Last Delivery Action Message:
    This option creates two different reports for success and failed bundles. The Success report lists all the bundles that were successfully installed on the devices in the last given (n) number of days. The Failed report lists all the bundles that failed on the devices in the last given (n) number of days.
    The deployed bundle status information is logged to the ZLM Server based on the Bundle Delivery Action Message. The messages are logged as Event Log Message, depending on whether the bundle was installed or failed to install on the agent. Once the bundle is installed or failed to install, the equivalent status message is logged to the server with its timestamp. By default , the logged messages are cleaned up periodically every 60 days or as configured for the MessageCleanup loaded module on the ZLM Server.

    Inventory Reports :

    The Inventory reports are dependent on the software and hardware inventory of the devices which are registered to the server. The inventory is updated to the ZLM server at a periodic interval of 24 hours (default) from the managed devices. The software inventory scan is also triggered when some new packages or bundles are changed on the device, or when any zmd settings or bundle lock information is changed. The zmd settings that controls the inventory scan and update schedule are:
    software-inventory-enabled, refresh-interval-software, refresh-interval-hardware, hardware-inventory-enabled and real-time-package-updates.
    The real-time-package-updates settings determines if the device need to send their inventory only at the periodic schedules or when the packages or bundles are changed on the device. The data reflected on the reports will be up-to-date and accurate for the last successful update of inventory to the server.
    So, it is important to ensure that these settings are properly configured for the devices in the management zone. These inventory refresh settings may lead to some performance overhead on the ZLM server if its value is less than the expected setting and when lot of devices simultaneously roll-up the inventory to its registered server.

    Option Reporting Title and Description
    9 Report Count of Packages installed per Device :
    This report displays the count of the total number of packages installed on the devices that are registered to the server, and whose software inventory was successfully updated. The count of packages installed cannot be seen collectively for every device. You must navigate to the inventory section of each device object in ZCC to see the details of the packages and the count.
    12 Report Devices Information whose Hardware or Software Inventory is not Updated to the Server:
    This option creates two reports related to software and hardware inventory of the device. The first report gives the details of the devices whose software inventory is not sent to the ZLM server for the first time. The second report lists the devices whose hardware inventory is not sent to the server for the first time. This inventory information is also necessary if you want to get the reporting statistics on the updates applicable to devices or for other reports which consume agents inventory data.
    20 Reports on Comparing Software Inventory Deviation Between Baseline Device and One or More Defined Devices:
    This option allows you to compare and report (textual / non html) the software package inventory of one or more devices which are registered to the server and if their inventory is updated. The device which is considered as reference for comparing its inventory is called the baseline device (device alias name). The devices whose inventory is to be compared with the given baseline device must be stored in a text file say, devices.txt file , with one device object name ( device alias name in ZCC) entry per line as follows:
    Device1
    Device2
    You must provide the absolute path of the text file when the script prompts for the filename to compare the package inventory. Each of the device in this file (devices.txt) is compared against the baseline device and the packages drift is computed and stored in the Device1.baseline-drift and Device2.baseline-drift files under the subdirectory called drift . The content of the file Device1.baseline-drift will be as follows:
    (-removed): zmd-inventory-0:7.2.2.0-0.0.i586
    (+updated,installed): zmd-inventory-0:7.3.0.0-0.0.i586
    This can be interpreted as: package = ‘zmd-inventory’ , ‘epoch: version-release.arch’ = 0:7.2.2.0-0.0.i586 was removed from the Device1 compared to baseline and it was updated/installed with package=’zmd-inventory’ and ‘epoch: version-release.arch’ =0:7.3.0.0-0.0.i586. Every package that is shown as ‘removed’ from a device may not be updated with its higher version. Sometimes existing package may be removed and new package can be installed /updated on the Device1 with respect to the baseline device.
    Similarly, you can analyze data for Device2.baseline-drift file. The report option provides additional information related to the ordering the devices according to the packages drift. The devices whose packages are drifted more from the baseline device are displayed below the devices that exhibit package drift of lesser magnitude. This result is stored into ‘devices_drift_order’ file under the drift directory created. The content of the file will be as follows:
    Device1 : 650 ( lesser drift)
    Device2 : 750 (higher drift)

    Note: The values 650 and 750 below are just numbers based on some internal calculations to get the order and does not hold any logical significance. There are some assumptions made to report as part of this reporting option.
    23 Report ZLM Agent Version Installed on the Devices Based on Agent Package inventory:

    This report lists the installed agent version based on the core agent package inventory information such as zmd, novell-zenworks-zlm-release packages. This may not be the best way to get the agent version. However, assuming that packages are not manipulated after installation of full ZLM agent, the results would give you an idea about the version of the ZLM agent running across all the existing managed devices registered to server. As part of this option, two reports are created in different ways to ensure that the results are comparable and identical.

    Device Reports:

    The classification of these reports is done based on the devices and allows you to generate device reports related to packages, bundles, updates, and device accessibility to the server.

    Option Reporting Title and Description
    10 Report Devices with Mandatory or Optional Package Updates:

    This report provides the information about the devices which has any mandatory or optional updates available. It creates two reports – one for devices having only optional updates, and the other for devices containing (at least one) mandatory update. The updates are available for a device when there is an update icon visible against the devices in ZCC. On clicking the update icon, it lists the package or bundle updates that is applicable to the device. The updates can be available for a device from any bundle when its packages are of higher version than the existing packages installed on the device. On assigning the bundle to a device this becomes a mandatory update from an optional one.
    13 Report Devices where given Bundle and Version is Installed:
    This report delivers the information about the devices where the deployed bundle with a given name and version are successfully installed. This reporting option also generates an additional report to show the count of devices where this bundle of deployed version is installed, once the software inventory of bundles reported to the server.
    Note : The software inventory from zlm7.3.0 onwards are updated everyday (by default) unless the zmd settings real-time-package-updates is enabled.
    18 Report Devices that are Inactive or not Contacted/Refreshed to the Server, for a given Number of Days:
    This report lists the devices which could not reach the server for periodic refresh actions. The device is fully refreshed to update the assignments/actions from the registered server everyday by default (refresh-interval) or partially refreshed if ZMD process wakes up from sleep mode. The managed device sometimes cannot reach the server either due to the service not being active/present or changes of its guid /network issues. You must check the ZLM agent device health status to confirm if this is active. The device on refresh action, updates its refresh timestamp to the server. The timestamp is based on UTC timezone as stored in the ZLM postgreSQL schema and must be converted to its local time before analysis.
    19 Report Devices where software package with given name and version is installed or not :
    This report list the devices where any package with given name, version and target is installed. The package name input is mandatory. The package version and target values are optional. The package name is matched along with version and target strings as given by the user to report those devices where this package is installed. This option also creates a report to list the devices where a given software packages name, version, target is not installed.
    29 Report devices whose last inventory scan timestamp information is not up-to-date on the Server :
    This report lists the devices whose inventory information has not been updated to the server as per inventory refresh settings and the time of last inventory scan is more than specified time. Such devices which might have updated their scanned inventory to the server in past but has not been updated since given period of time. It helps to trace devices whose inventory data sent as part of periodic scan that might have become stale. The devices are listed in order of its type i.e Server, Workstation, ZENworksServer and the last inventory scan timestamp.

    General Reports or Utility:

    This category covers some of the reports that are related to general use case and are different from the above categories.

    Option Reporting Title and Description
    15 Creates all of the reports automatically from Report Options 1 to 12 in the utility :
    This option allows you to automatically generate the reports from the Report option 1 to option 12 listed in the utility.
    16 Report Last Delivery status (success/fail) for a given policy based on message logged per Device :
    This option reports the last delivery status of a given policy, which is deployed on the device(s) for its enforcement, as per the schedule. Once the policy schedule is triggered and the policy is enforced successfully or failed to apply on the device, the Policy Enforcement Action Message is centrally logged to the server for the corresponding device as an ‘Event Log Message’. If you are logged into ZCC, this message notifies the ZCC user if the policy was enforced on the device. This option generates two reports to list the devices where a given policy was deployed and got enforced successfully or devices where it failed to apply.
    17 Report Last Delivery status (success/fail) for a given bundle based on the message logged per Device :
    This option reports the last delivery status of a given bundle, deployed on the device(s) for its installation, as per the configured schedule . Once the bundle schedule is triggered and bundle is installed or failed , the ‘Bundle Install Action Message’ is centrally logged to the Server for the respective device as an Event Log Message. This message notifies the user if the bundles were installed on the device. This option generates two reports to list the devices: One report is to list devices on which the bundle is successfully installed. The second report is to list the devices on which the bundle failed to install.
    21 Triggering Package Updates manually for Bundles in a given folder :
    This option is a tweaking method which allows you to trigger package updates internally in the ZLM schema for the bundles by creating the required Queue Actions. You must provide the ZCC folder name with the path, which contains the bundles. The package updates are computed for all the bundles under this folder or its sub folders.
    This action recomputes the updates information for the bundles and the packages it contains. This is a costly operation which is done first time when the bundle is created with packages and need not be performed when the server is busy. This action is performed by the ZENloader process of the ZLM and status of these actions are reported to /var/opt/novell/log/zenworks/loader-messages.log file on the server. The option does not create any report but ensures that the software update details in the various reports are up-to-date.
    22 Triggering Device Updates manually for Devices in a given folder :
    This option is similar to the last option which allows you to Trigger device updates internally in the ZLM schema for the all the devices by creating the required Queue Actions. You must provide the ZCC folder name with the path- which contains the devices and also the device type (Server/Workstation).
    The device updates are computed for all the devices of given type under the specified folder path.
    This action recomputes the updates information for the devices. This is a costly and cpu intensive operation, done first time when the bundle/bundle group is assigned to any device or device group ,or when a modified software inventory is updated to the server. It is recommended not perform this operation when the server or ZENloader is busy. This action is performed by the ZENloader process of the ZLM and status of these actions are reported to /var/opt/novell/log/zenworks/loader-messages.log file on the server. The option does not create any sort of report but ensures that the software updates details in various reports are up-to-date.
    27 Report the ZMD setting value configured on each device for a given setting name :
    This report lists the value of a given ZMD setting name on each of the existing devices in the zone. If the input setting name is log-level , its configured value on each of the device is displayed the corresponding device. This report allows you to confirm if the settings are consistent on all the devices in the management zone. The ZMD settings are updated as part of the software inventory of the device, So the software inventory setting software-inventory-enabled should be enabled to get a valid and up-to-date report.
    30 Report the queue action details that failed or are pending for execution by ZENloader :
    This report lists the Queue Actions (related to bundle, device, assignment, updates, refresh etc) that failed or are pending for execution in the loader queue. The actions can fail sometimes either because ZENloader is unable to handle the load of processing multiple actions in its thread due to lack of resource (memory/cpu) or because actions are very costly to operate. This can help administrator to track and troubleshoot loader related issues.
    31 Purge the successfully processed ZENloader queue actions which are older than the given number of days :
    This reporting option allows the you to delete the Queue Actions from the ZENQueue which have been successfully processed and are older than the given number of days. The actions once processed in the Queue cannot be used again. These records piled up over a period of time, thus increasing the size of the table which might also impact performance over a period of time. So such stale records can be cleaned up from the table by using this option thus reducing the table size.

    Configuration Steps

    Instructions :

    • The reporting utility allows generation and collection of html based reports under a given directory based on the selected option under the utility menu.
    • The server preference compute-package-device-updates should be enabled on the ZLM server and inventory should be consistently/periodically updated from agents to get accurate report data.
    • The /root/.pgpass should be configured with the db connection password as present in the /etc/opt/novell/zenworks/hibernate.cfg.xml file on the ZLM Server.
    • Updates for packages inside the catalog are excluded in ZLM and the query results entirely depend on the software inventory data for the devices, packages, bundles, and updates present in the database.
    • The html reports created for a given option are stored under the input directory say /tmp/reports that is created based on the current timestamp.
      Example: /tmp/reports/report-2010-03-16-18:47/zlm-report.10551-10.html
    • The unique report file created under a timestamp based directory has the name and suffix as the selected option.
      For example, for Report Option 10, the file will be zlm-report.10551-10.html
    • If errors such as “Failure : Could not fetch the information from database” occur, perform the following steps for the configuration and verify if the database can be queried from the location and user from where the script is executed.
    • The script must be executed as a root user on the ZLM Primary Server.

    Steps :

    • Set the /root/.zlmanrc file to username as administrator and its password to the skip interactive prompt.
    • The database user in the script is ‘zenadmin’ and database name is ‘zenworks’ by default.
    • Ensure that the /root/.pgpass file is set with database connection.password value (e.g. 51d91c9851384e44848fb769b4b7195c) from the /etc/opt/novell/zenworks/hibernate.cfg.xml file.
      For example, If ZLM server has embedded postgreSQL database , the /root/.pgpass file should be localhost:5432:zenworks:zenadmin:51d91c9851384e44848fb769b4b7195c
    • Provide the following test query as ‘root’ to check if the database (local/remote) can be accessed from the ZLM server.
      Test DB Query: psql -d zenworks -U zenadmin -c “select count(*) from os_targets” -t
      Result should be some finite numeric value, say 43.
      If you are prompted for the password for the zenadmin(user), ensure that in the /root/.pgpass file, the value 127.0.0.1 is changed to localhost if the database is local postgreSQL.
    • Open the script zlm-report.sh and modify the values for the following configuration variables applicable to the ZLM server environment.
      pg_DbHostIP=IPAddress_Postgres_Device
      ZLMMASTER=ZLM_Server_IPAddress
      ZLMZONE=ZLM_Management_Zone
      EDIRPASSWD=ZLM-Server-Admin_OR_Tree-Password
    • Run the script as root user on the ZLM Primary Server with PostgreSQL as datastore , as per the usage syntax given in the following step.
    • Provide the mandatory option -d , the directory to store the resulting reports. All the reports are stored under this path based on the timestamp based directory names and the selected option.

    Appendix :

    Package Updates : RPM package update for the bundle packages that are installed on the device.
    Device Updates : All packages that are available as software updates for a given device either through a bundle or independently.
    Mandatory Updates : Packages from the bundle or bundle group which are available as software updates for the given device after its mandatory assignment.
    Optional Updates : Packages from the bundle or bundle group which are available as software updates for the given device without any assignment. These are low priority available updates.
    Inactive Devices: Devices which have not contacted the server during refresh for a given time period.
    Delivery Status: The status (Installed/Failed) of bundles or policies that are assigned to the devices.
    Baseline Device: The device whose installed packages are compared with the packages of another device to measure its drift or deviation from baseline.
    Inventory Drift: Measure the deviation of software packages (name:epoch-version-release.arch) that are installed on a baseline device against the specified devices.

    It is recommended not to modify the reporting content and the options in the script unless you have a complete understanding of the script and the ZLM schema.

    Solution

    BashScript : zlm-report.sh
    Usage: sh zlm-report.sh –d directory-to-store-reports
    Input is the Directory name where all the html reports will be stored .

    Example: sh zlm-report-latest.sh -d /tmp/reports

    Click to view

    ZLM Reporting Utility Options

    If you have any questions or suggestions related to the reports or the query, you may contact tarvindkumar [at] novell [dot] com.

    Acknowledgments :

    Reviewer: Shubhashree D – Technical writer, Novell, Inc.

0 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 5 (0 votes, average: 0.00 out of 5)
You need to be a registered member to rate this post.
Loading...Loading...

Tags:
Categories: Uncategorized

Disclaimer: This content is not supported by Novell. It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test it thoroughly before using it in a production environment.

2 Comments

  1. By:WilliamByrne

    Very nice.

  2. By:xavieroneill

    Thanks, this has saved me an awful lot of time!

Comment

RSS