Novell Home

Using Webalizer to Analyze Novell BorderManager Logs

Novell Cool Solutions: Trench
By Chris Bailey

Digg This - Slashdot This

Posted: 5 Feb 2003
 

In our district, we are currently having issues with having enough bandwidth to go around. In order to justify the additional bandwidth, I need to show the administration what our Internet access was being used for -- such as which sites were being visited.

The default BorderManager log files are aren't anywhere near adequate to present to a non-technical school administrator. We needed something that could tell us more information, yet in a way that someone else could read it. Webalizer, an open-source project, seemed to fit the bill for our situation. Webalizer is able to display statistics as the number of hits, bandwidth used, as well as the top URLs visited by your employees or students. Ulrich Neumann has been kind enough to port Webalizer to the NetWare platform.

Some prerequisites must be completed before we can proceed with the setup of Webalizer.

  • NetWare 5.1 SP5 or NetWare 6.0 SP2
  • Novell BorderManager
  • Perl installed on the NetWare server
  • A copy of Webalizer, which can be downloaded at:
    www.mrunix.net/webalizer
  • Novell Enterprise Web Server or Apache running on your BorderManager server.

The first step will be to install the Webalizer software. Extract the software from the zip file, and copy the files contained in the SYSTEM folder to the SYS:\SYSTEM folder on the BorderManager server. You will then want to copy the webalizer.nlm file from the language directory of your choice to the SYS:\SYSTEM folder on the server as well. Next, copy the files from the ETC folder to the SYS:\ETC on the server. (These were the installation instructions as of January 2003 -- read the readme.txt file included with your download to verify the Webalizer installation process.)

You will now need to edit the SYS:\ETC\WEBALIZER.CONF file with the settings specific to your system. Some specific lines to change include:

  • LogFile -- REM this line out by placing a # as the first character of the line. I will detail the reason why later in this document.
  • OutputDir -- enter the path to a directory under your web server root for the generated report. For example, I have created a directory called STATISTICS underneath our web server root.

    OutputDir /apache/nwdocs/statistics
  • HistoryName -- unREM this line by removing the beginning # character.
  • Incremental -- unREM this line, and change the setting to "Yes".
  • ReportTitle -- change the following text to something significant to you.
  • HostName -- correct the host name displayed as appropriate.

Save the SYS:\ETC\WEBALIZER.CONF file.

In order to account for all of the log files, some setting changes need to be made to the proxy server within BorderManager. Load NWAdmin32 and enter the BorderManager Setup properties tab for the server. Select "HTTP Proxy" and click "Details?". Go to the "Logging" tab.

Take note of the settings displayed on this screen. For Webalizer, we will use the Common logging format. The "Log File Directory" and "Log Rollover" settings should also be set exactly as detailed above -- otherwise, you will have problems with the Perl script detailed later in this document. Once these settings are confirmed, click OK and close out of NWAdmin32.

Webalizer is designed with a single log file in mind. However, BorderManager is designed to create separate log files, possibly divided by date or by file size. To work around this problem, the settings in NWAdmin32 are set for one log file per day. We then utilize a simple Perl script to tell Webalizer which log file to analyze for the day.

Create the Perl script by pasting the text below into a file named SYS:\SYSTEM\BMLOG.PL.

#!/usr/bin/perl 
 use Time::Local; 
 my ($y, $m, $d) = (localtime)[5,4,3]; 
 $y -= 100; 
 $m += 1; 
 if ( $m < 10) 
 { 
 $aftermonth = "/etc/proxy/log/http/common/0" . "$y" . "0" . "$m"; 
 } 
 else 
 { 
 $aftermonth = "/etc/proxy/log/http/common/0" . "$y" . "$m"; 
 } 
 if ( $d < 10) 
 { 
 $afterday = "$aftermonth" . "0" . "$d" . ".LOG"; 
 } 
 else 
 { 
 $afterday = "$aftermonth" . "$d" . ".LOG"; 
 } 
 $commandline = "webalizer " . "$afterday"; 
 system "$commandline"; 

NOTE: The Perl script I programmed for this project has a major limitation. In order to analyze the correct log file, the Perl script must be run at 11:59 p.m. on the same date as the log file to be analyzed is named. For example, the log file for January 2, 2003 is named 030102.LOG. In order to be analyzed properly, the Perl script uses the current date to generate the correct log file name, thus the reason why it must be run at 11:59 p.m. the same day as the log file name. I am not a Perl programming expert, nor have the time to become one, so if anyone would like to contribute a new Perl script for this purpose, please do so.

Finally, you will want to use CRON or NetWare Remote Manager (the "Schedule Tasks" feature) to schedule the command "PERL SYS:\SYSTEM\BMLOG.PL" to run every night at 11:59 p.m. You can also run Webalizer manually by utilizing the following command format, replacing the date information as appropriate.

Webalizer /etc/proxy/log/http/common/YYMMDD.log

The reports will be stored in the OutputDir directory you specified in the webalizer.conf file.

Advisory Board Member Chris Bailey is a member of the School Cool Solutions Advisory Board. You can find him in the wee hours of the evening in the School Cool Forums.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell