Interoperation Guide - NBM 3.8.10 VPN Client and Racoon Server
Novell Cool Solutions: AppNote
By Barochia Bhavatosh
Digg This -
Posted: 29 Dec 2005
Interoperation Guide - NBM 3.8.10 VPN Client and Racoon Server with Xauth-PSK and Xauth-Hybrid methods
The NBM 3.8.10 VPN Client can be used to connect to any Virtual Private Network (VPN) Server capable of authenticating, using Extended authentication (Xauth) in the main and aggressive modes of IKEv1. This Appnote considers Racoon as an Xauth-capable VPN server and highlights the Xauth support provided by the NBM 3.8.10 VPN Client.
NBM 3.8.10 VPN Client Features
The NBM 3.8.10 VPN Client supports the Xauth-Pre-Shared key (PSK) method and Xauth-Hybrid method in the Aggressive mode. Xauth-PSK support is provided in the Main mode. These modes are described in more detail later in the article.
About IKE Xauth (from the RFC)
"Internet Key Exchange protocol (IKE) Extended Authentication (Xauth) is a draft RFC developed by the Internet Engineering Task Force (IETF) based on the Internet Key Exchange (IKE) protocol. The Xauth feature is an enhancement to the existing Internet Key Exchange (IKE) Protocol feature. Xauth allows authentication methods to perform user authentication in a separate phase after the IKE authentication phase 1 exchange."
"The Xauth feature is an extension to the IKE feature, and does not replace the IKE authentication."
The Xauth-PSK Method
The Xauth-PSK Method is a type of authentication in which pre-configured secrets are used for client extended authentication. Xauth-Hybrid method is a type of authentication in which certificates are used for client extended authentication.
There are two basic modes of establishing an authenticated key exchange, namely:
- Main Mode: A mode of establishing ISAKMP SA in which the two peer identities are not revealed.
- Aggressive Mode: A mode of establishing ISAKMP SA with a fewer number of message exchanges. This mode does not protect the identities of two endpoints. Both the modes generate authenticated keying material from an ephemeral Diffie-Hellman exchange (DH group).
Main mode negotiation is more secure than Aggressive mode negotiation. For more information, see RFC 2409 - The Internet Key Exchange.
Racoon is the Internet Key Exchange (IKE) daemon for automatically keying IPsec connections. The Racoon server must be installed and configured for Xauth-PSK and Xauth-Hybrid methods, in both Aggressive and Main modes, before connecting to the NBM 3.8.10 VPN Client.
To install the Racoon server, you will need IPsec-tools. These include racoon, libipsec, setkey, and racoonctl. For more information on the IPsec tools, visit http://ipsec-tools.sourceforge.net/
Installing Racoon Server
Install the Racoon server as follows:
1. Download CVS version of IPsec-Tools from:
2. Enter the following command at the console:
cvs -z3 -d:pserver:email@example.com:/cvsroot/ipsec-tools co -P ipsec-tools
For more information on this command, visit:
3. Change the directory to the ipsec-tools directory.
4. Run ./bootstrap
Note: Ensure that the permission for bootstrap is executable enabled. If it is not, use chmod to enable execute permission.
5. Run ./configure --enable-hybrid --enable-natt
6. Run make.
7. Run make install. The binary will be installed on $prefix/sbin/racoon. Here, $prefix referes to the exact pathe, for example, /usr/local/sbin/racoon.
Configuring Racoon Server for Xauth-PSK and Xauth-Hybrid Methods in the Aggressive Mode
In order to configure the Racoon server for Xauth-PSK and Xauth-Hybrid methods in the Aggressive mode, you must edit the racoon.config file located at: <path/to>/raccoon.config.
A sample raccoon.config file is given below:
Figure 1: Sample Racoon config file
In the above file:
- "/path/to/psk.txt" refers to the path of the psk.txt file. Enter the actual path to the location of the psk.txt file here, for example, /etc/raccoon/raccoon.config.
- "/path/to/certs" refers to the path of the psk.txt file. Enter the actual path to the location of the psk.txt file here.
Remote Anonymous Connection
The remote anonymous connection type accepts connection from any client, unless a client already has a separate connection in the .conf file. Here, the connection is established through the proposal included in the block for anonymous connection.
Hostwise Remote Connection
The remote 22.214.171.124 connection type accepts connection from the client coming from the 126.96.36.199 host. Here, the connection is established through the proposal included in the block for 188.8.131.52 connection. In both anonymous and 184.108.40.206 connections, the exchange mode will be Aggressive.
In the above raccoon.config file, 'Anonymous' indicates that the connection is meant for any client connecting to the Racoon server in the Xauth-PSK mode.
Before you start the Racoon server:
1. Edit the psk.txt file. A sample psk.txt file is given below:
220.127.116.11 abc 18.104.22.168 def 10.160.94.3 mekmitasdigoat 172.16.1.133 0x12345678 22.214.171.124 whatcertificatereally firstname.lastname@example.org mekmitasdigoat foo.kame.net hoge
In the psk.txt file, all pre-shared keys related to clients are stored in the above format.
2. Run "racoon -f /<path/to>/racoon.conf" to start the Racoon server.
In the above raccoon.config file, '126.96.36.199' indicates that the connection is meant for the client connecting to the Racoon server in the Xauth-Hybrid mode and whose IP address is 188.8.131.52.
3. Get the X.509 certificate in the .pem format along with the key for extended authentication on Racoon server. Note: Any user certificate from Netware OS in .pfx format can be converted into .pem format, using the openssl tool.
4. Copy the certificate in the specified path given in the racoon.conf file; for example, "/etc/racoon/certs"
NBM 3.8.10 VPN Client Configuration for Xauth in the Aggressive Mode
1. Start the NBM 3.8.10 VPN Client.
2. Click the Configuration tab and select Xauth-PSK from the Authentication method drop-down list.
Figure 2: Xauth-PSK authentication
3. Click the VPN tab and enter the following information:
- VPN server ip address: IP address of Racoon server
- Username: Any name other than the name of root which has login access to the Racoon Server
- Shared Secret: Secrets associated with the client's IP address on the Racoon Server
Figure 3: VPN information
4. Click Policy Editor. The Policy Editor dialog appears.
5. Select Aggressive Mode from the IKE mode drop-down list.
Figure 4: Aggressive Mode
6. Check the Use My Policy check box.
7. Click OK to establish the connection. The following dialog appears on successful connection:
Figure 5: VPN statistics
1. Get the trusted root certificate of the certificate that has been setup in the Racoon server, in the .der format.
2. Copy the certificate to the following Windows directory of the machine from which the connection is being established to the Racoon server:
3. Start the NBM 3.8.4 VPN Client.
4. Click the Configuration tab and select Xauth-Hybrid from the Authentication method drop-down list.
Figure 6: Xauth-Hybrid authentication
5. Click the VPN tab and enter the following information:
- VPN server ip address: Racoon server's IP address
- Username: Any name other than the name of root which has login access to the Racoon server
- Password: Password of the user
Figure 7: VPN information for Xauth-Hybrid
6. Click Policy Editor. The Policy Editor dialog appears.
Figure 8: Policy Editor dialog
7. Select Aggressive Mode from the IKE mode drop-down list.
8. Click OK.
9. Check the Use My Policy checkbox.
10. Click OK to establish the connection. On successful connection, the following dialog appears:
Figure 9: VPN statistics
Racoon Server Configuration for Xauth-PSK Method in Main Mode
To configure the Racoon server for Xauth-PSK and Xauth-Hybrid methods in the Main mode, you must edit the racoon.config file. A sample Racoon configuration file (racoon.conf) is given below:
Figure 10: Sample Racoon config file
Connection details are similar to the details provided for the racoon.conf file in the Aggressive mode. Entries of pre-shared keyes in psk.txt file are similar to the entries provided in the psk.txt file, in the Aggressive mode. But here, the exchange mode for both anonymous and 184.108.40.206 connections is Main.
NBM 3.8.10 VPN Client Configuration for Xauth-PSK Method in the Main Mode
1. Start the NBM 3.8.10 VPN Client.
2. Click the Configuration tab, then select Xauth-PSK in the Authentication method field.
Figure 11: Xauth-PSK authentication, Main Mode
3. Enter the following information:
- VPN server ip address: IP address of the Racoon server
- Username: Any name other than the name of the root which is has the login access to the Racoon server
- Password: Enter the user password
- Secret: Secrets associated with that client's IP address on the Racoon Server
Figure 12: VPN information
4. Click Policy Editor. The Policy Editor dialog appears.
5. Select Main Mode from the IKE mode drop-down list.
Figure 13: Main Mode
6. Click OK.
7. Check the Use My Policy checkbox.
8. Click OK to establish the connection.
Some of the Administrative commands are given below.
- See the Logs: tailf /var/log/messages
- Start Racoon server: racoon -f /path/to/racoon.conf
- Flush network policies on Racoon server: setkey -F
- Check the status of existing network policies: setkey -PD
- Dump debug messages on activities of pushed security associations such as SA deletion: setkey -x
The following scenarios have been tested:
- Racoon Server Platform : SuSe 9.2
- NBM 3.8.4 VPN Client Platform : Windows XP SP2,Windows 2K, Windows 98
- NBM 3.8.4 VPN Client Version : 3.8.10
- Racoon Server Version : CVS Version
Note: The current version of ipsec-tools, Version 0.6.1, does not support the xuath-psk and xauth-hybrid methods. Check the NEWS file of the Future releases for availability of these features.
The Xauth feature adds value to the interoperating capability of the NBM 3.8.10 VPN Client with any Xauth capable VPN servers. You can connect the NBM 3.8.10 VPN Client to any Xauth capable VPN server in the following modes:
- Xauth-PSK and Xauth-Hybrid methods in Aggressive mode
- Xauth-PSK method in Main mode
For more information on Novell Border Manager, visit the Novell documentation Web page at:
Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com