iPrint or Apache is slow or unable to accept new jobs

  • 7012473
  • 24-May-2013
  • 14-Aug-2013

Environment

Novell iPrint for Linux
Novell Open Enterprise Server 2 (OES 2) Linux
Novell Open Enterprise Server 11 (OES 11) Linux

Situation

Users either complain about print jobs either taking a long time to be accepted or print, or new print jobs cannot be submitted.  

A list of symptoms that "may" be reported and/or observed include:

  • "iPrint and Apache are slow".
  • Sending print jobs via iPrint is slow.
  • Listing printers within an application is slow.
  • LAN trace shows some Windows workstations excessively querying the print manager over the iPrint Apache port 631.
  • The following command shows dozens of Apache processes running: ps -ef | grep httpd
  • There are one or more httpd2-worker threads showing high CPU utilization (via top)
  • The iPrint /psmstatus page will show a "healthy" iprint manager status.

Resolution

Follow the steps in the "Additional Information" section of this TID to identify which workstations are excessively querying the Print Manager.   The Options listed below will need to be applied to each workstation which is identified as being an offending workstations. Be prepared to find that more than one workstation contributing to the problem.   Note:  So far, this has been seen only from Windows XP workstations.
 
Option 1: Update to the 5.92 or later iPrint Client

Option 2: Remove the HP Toolbox utility
Follow the steps in the "Additional Information" section of this TID to identify which workstations have the HP Toolbox software installed, and remove that software tool. 
The steps will likely need to be repeated on each workstation that has this utility installed.

Option 3: Turn off the "alert" feature of the HP Toolbox utility
The HP Toolbox utility runs as a service on Windows and depending on the version, may be started with the following registry entry:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run]
"ToolBoxFX"="\"C:\\Program Files\\HP\\ToolBoxFX\\bin\\HPTLBXFX.exe\" /enum:on /alerts:on /notifications:on /systrayIcon:on /fl:on /fr:on /appData:on"

It has been observed that changing the "/alerts:on" to "/alerts:off" parameter on that line, will stop the excessive requests to iPrint.  It is unknown, however, the full effect it will have on locally attached devices.

Cause

Some HP printer drivers install a side program called the HP Toolbox (hptlbxfx.exe) when configuring a local printer (viz. non-iPrint printer).  HPTLBXFX causes the Windows Print Spooler to aggressively query printer job lists -- at times in excess of a query every second, for each printer installed.  So if you have 4 iPrint printers installed, and a local printer; the iPrint server will see in excess of 4 queries per second from just this one workstation.  Multiply this by a handful of machines, and the result is thousands of "junk" requests coming into the Apahce web server (which relays the requests to the Print Manager) -- every minute.  Apache becomes overwhelmed and the symptoms above occur.

Status

Reported to Engineering

Additional Information

The following technique can be used to diagnose this problem.
  • Capture an IPP filtered LAN trace  from the iPrint server
    • At a terminal session (putty or otherwise), run this command    tcpdump -s 0 port 631 -w /tmp/ApacheiPrintTraffic.cap
    • Let tcpdump run during high volume printing times and/or when this matter is occurring.  3-5 minutes of capturing should be sufficient.
    • After 3-5 minutes, press  Ctrl + C  to stop the packet capture.
  • Review the ApacheiPrintTraffic.cap file using a network analysis tool, such as WireShark.
  • Look for workstations sending 50+ IPP requests per minute.  A quick way to do so (in WireShark) is to:
    • click the menu option Statistics -> Conversation List -> IP v4
    • in the table that comes up, sort on the Duration column in descending order (so the most talkative workstations are first)
    • note the IP addresses of the workstations that have a duration close to the length of the packet trace.
      In a 5 minute (or 300 second) packet trace, the troublesome workstations will likely have a duration of 298 seconds or more.
    • in the "Filter:" box (below the icons), enter the following and click the "Apply" button:
          (http.request.full_uri) && ip.addr==IP_ADDR_FROM_above
      NOTE: the IP_ADDR_FROM_above is one of the ip addresses from the previous step. 

      At the bottom or the WireShark window, you will see the name of the packet trace and in the next section, the # of packets in the trace and the number of "Displayed" packets -- which is packets that met the filter criteria.  This will be the number of request this one IP address sent to the iPrint server.  You can use this number to calculate the number of requests per minute (or second)
    • if the # of displayed packets is >= the # of seconds of the packet trace, investigate that workstation and look for the hptlbxfx software and apply one (or more) of the options in the "Resolution" section.
    • repeat this section for each high duration workstation in the packet trace.
  • Wait an hour (or a day) and repeat this process to verify there are no other devices generating superfluous requests.