Novell is now a part of Micro Focus

BorderManager Troubleshooting Tips

Novell Cool Solutions: Feature

Digg This - Slashdot This

Posted: 3 Nov 2005

General Troubleshooting Tips

Troubleshooting BorderManager servers requires a basic understanding of how the various components and features work, in theory and on NetWare.

First, you need to isolate the problem.

  • Filtering problem - UNLOAD IPFLT, or use Filter Debug techniques
  • Browser problem - try other browsers, other PCs
  • Proxy problem - try bypassing the proxy
  • Access Rules problem - try not enforcing rules
  • Hardware problem - try a different cluster node, reboot, check hardware statistics in various monitoring programs
  • Routing problem - occurs especially on VPN

Troubleshooting Filtering

Assumption: You are filtering something you want to allow, and you need to find out what it is.

Initial Technique: UNLOAD IPFLT - if the app starts working, you know you need a filter exception.

Using the Filter Debug Technique

Edit T1.NCF, and use these commands 

More Sophisticated Filter Debug

Use this if you suspect a virus problem. Manually clear the logger buffer (F5 key), then edit T2.NCF with following commands:


This will enable filter debugging for only 1 second at a time - 2 seconds has proven to be too long in some cases.

Troubleshooting Proxy Issues

Try Different Browsers, including Firefox, and on different PCs Numerous problems with accessing certain web sites via proxy go away with Craig?s PROXY.CFG file (Tip #63 at Craig's web site) UNLOAD PROXY, LOAD PROXY ?CC, to clear cached data (especially if abends are happening) Try bypassing proxy with a PC on public subnet or via filter exceptions to isolate issue to proxy Use RTMonitor or analyze log files to see if you are browsing to the URLs you expect (important with access rules) If Proxy does not load at startup, you probably need to delay STARTBRD until NLSLSP is completely initialized. (May need to unload ACLCHECK and reload it along with PROXY if you have had this issue) Disable TLS support on browsers if having SSL Proxy Authentication issues

Troubleshooting VPN

7 Tips for VPN Troubleshooting

1. Use the IKE Screen

Unacknowledged messages mean the other VPN server is rejecting the connection. Look at other server to try to determine the problem. Also, ACL Check failures are frequently due to misspelled subject names when configuring the VPN - the audit logs will show you what the subject name needs to be (hint: Copy and Paste).

2. Use Novell Remote Manager (https://x.x.x.x:8009).

The audit log shows what the VPN server sees.

3. Use the CSAUDIT command.

This command shows audit logs on the server console.

4. Use the _VPN command.

This shows some useful VPN debugging commands.

5. Use the RUNVPN -L3 command.

This command brings up a debug window showing some useful data. It should also restart Java SCM process, if you manually killed that process.

6. Use the UNLOAD IPFLT command.

Failure of stateful filter exceptions (can be on either side) can keep the Master from configuring the Slave. This typically seems to be a problem with the IPX/TCP-ST exception dropping return packets.

7. Disable RIP

Site-Site VPN fails if RIP advertises the tunnel. Filters need to be active, or RIP needs to be disabled.

Troubleshooting VPN C2S Issues

Client-to-Site issues can occur in many places. Here are some examples:

  • C2S VPN software version on the client
  • Other VPN clients may conflict with Novell's Personal Firewall on the client
  • Router in use at the client site (see tip #46 at Craig's website)
  • MTU size issues, particularly with internal WAN links involved
  • Routing issues, sometimes due to poor choice of IP addressing used for LAN or VPN Tunnel (conflicts with client addressing)
  • NMAS failures (patch the NBM 3.8 server)

Note that the Compatibility mode has no concept of traffic rules per user - it melds them all together in the IKE/SKIP environment.

Other Problems

Poor Internet Performance

The Korga virus and Welchia virus infected PCs, sending massive traffic to BorderManager servers. This used extra bandwidth, possibly causing high utilization. This was especially true if the Welchia virus was combined with an ICMP-ST filter exception.

Slow or No Internet Performance After Period of Time (usually a couple of days)

This could be cause by a stateful filtering bug. Performance recovers when you run UNLOAD IPFLT, LOAD IPFLT and patch the server. Or the problem may lie with the SCMServiceConfiguration java process - kill the process and restart it as needed with RUNVPN.

Failed Receiving Server DH Public Value

This is a Client-to-Site VPN issue that can be fixed with patch. If the NMAS authentication method is not "Logged", this problem may occur.

Transparent Proxy Not Working for SSL

  • Enable tunneling control in PROXY.CFG file.
  • Configure port 443 in PROXY.CFG file.
  • Configure port 443 in Transparent HTTP Proxy configuration in NWADMN32.

Can't Access HTTPS Sites (such as :8009, :52443)

  • Enable tunneling control in PROXY.CFG file.
  • Specify port numbers to tunnel (8009, 52443, etc.).

NBM 3.8 S2S VPN Changes Not Completing

  • Try rebooting the server.

S2S VPN Slave Not Configuring

Try using filter debug techniques to see if TCP port 213 traffic (or other VPN traffic) is being discarded. Or, you can unload IPFLT for 20 minutes.

NBM 3.8 VPN Server Won't Start IKE

There may be a missing attribute on server object for VPN. In ConsoleOne, choose VPN server object > Properties > Other. Look for vpnServiceIdentityDN. It should have a value of scmServiceObject.<ou>.<o> where <ou>=Server OU, and <o>=Organization.

NBM 3.8 S2S Slave VPN Server Won't Start IKE

A design issue resulted in the requirement for slave servers to be able to access a copy of the Root replica to start VPN.

Can't Sync eDirectory Without VPN - Need to for VPN

If you need to sync an NDS change to a slave in the same tree, set up an IPX tunnel between servers, or use a temporary VPN server (perhaps with Linksys VPN router).

Server Hangs When Unloading Proxy

Yes, this is still a problem, but Craig's PROXY.CFG helps.

TIMESYNC Waiting to Initialize at Bootup

This issue is fairly normal.

  • Enable IPX and add a SERVERID command in AUTOEXEC.NCF to allow the server to initialize TIMESYNC using IPX. You do not have to bind IPX to a network card.
  • Try XNTPD on NetWare 6.5 - see TID 10084753
  • Unload / Reload TIMESYNC in AUTOEXEC.NCF if necessary.

Loss of Filters / Filter Corruption

Make a backup BEFORE you have problems! If you are using FILTCFG, back up the SYS:ETC\FILTERS.CFG file. Otherwise, use FILTSRV_BACKUP_FILTERS <filename> and then back up the file. Once the problem occurs, use the FILTSRV MIGRATE -CF procedure, which relies on an accurate filters.cfg file.

Getting Free Help

Novell Public Forums:

NNTP (best) - or HTTP

Craig's Web Site:

Novell Knowledgebase and Documentation:

Cool Solutions:

Additional Resources:

Lite Book and Novell docs on BorderManager 3.7 / 3.8 Product CD

Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions.

© Copyright Micro Focus or one of its affiliates