Article

(View Disclaimer)

Introduction
Setting Up Wireshark
Finding the "Hello Package" and Server Certificate
Extracting the Private Key
Configuring Wireshark for SSL Decryption
Important Note for Firefox Users

Introduction

In many cases, it is a good idea to troubleshoot an existing setup using network traces. Using tcpdump on SLES and pktscan.nlm on NetWare, you can easily capture traffic in all the corners of your environment. These traces will tell you exactly who is sending information to where and what is contained in the information.

That last part can become tricky when SSL has been enabled. You can disable SSL on that connection and send the information in clear text over the wire to troubleshoot, but this might not be possible for security reasons, it can be a lot of work to reconfigure and maybe the error you're chasing is somehow related to the SSL session. Another option is to capture the traffic when it is encrypted with SSL and use Wireshark to decrypt the messages and make the information readable, so nothing has to be reconfigured in your environment.

To illustrate in this AppNote, I use SSL communication when logging in to the Identity Server. This is a HTTPS connection running on port 8443. Please also note that the example here is HTTPS; other protocols like LDAP will also work to decrypt.

Setting Up Wireshark

The first thing that you need is Wireshark. This can be obtained from the download page on the Wireshark website. Make sure you install the version that has the SSL decryption functionality. At the moment of writing the latest one (version 0.99.5) has this build in by default. If you do not see the "RSA keys list" and the "SSL debug file" fields described later in this document, you don't have Wireshark with the SSL decrypt functionality.

Another useful tool is OpenSSL. You'll need this because you need to make the private certificate readable by Wireshark. OpenSSL can also be easily downloaded from the Internet.

The network trace you want to analyze should include the SSL handshake. This means you need to start capturing when the client service requests a session with the other service. You can quickly verify this by looking to the trace. To do this, you first need to find the right packet where the handshake is taking place. When you're analyze traffic on a standard SSL port (like https/443 or ldaps/636), you can type "ssl" in the Filter field in Wireshark and press Enter. If you're not using a standard port, you need to tell Wireshark how to interpret the port. You need to filter the traffic based on IP address and port and set Wireshark to decode this as SSL. Then you can also use "ssl" as the filter to find the SSL handshake packets.

In the example below, I create a filter to find traffic going to the Identity Server on port 8443 (tcp.port==8443). The packets are not recognized by default as SSL, so I right-click on a packet and select 'decode as'. Then I click the transport tab and select SSL. After I've done that, I can see that the packets contain SSL information.

Click to view.

Figure 1

Figure 1 - Selecting SSL Transport

Finding the "Hello Package" and Server Certificate

The next step is to find the server "hello package" containing the server certificate. This package is part of the SSL handshake, and we'll use information from this package to set up the SSL decoding. The information you need to remember from this package is the source IP address, the source TCP port, and the Subject Name of the server certificate. Also, the serial number can be used to double-check whether you have the right certificate.

In the following screenshot you can see the server "hello package;" it is the response to the client "hello". You can find both packets in the beginning of the SSL session. The information we need from the packet is the source IP address 149.44.38.230, the src TCP port 8443, and the subject name from the certificate or id-at-commonName - "nam-idp.lab.ba".

Click to view.

Figure 2

Figure 2 - SSL packet information

Extracting the Private Key

Next, we need to find and extract the private key from the server. Wireshark will need to use this to read the handshake and get the information for decrypting the SSL packets.

1. In the Access Management console, go to Certificates.

2. Find the certificate with the same Subject Name as found in the server "hello package."

3. Open this certificate and click Export Private/Public Keypair.

4. Save the certificate to a file on the local hard drive.

5. Provide a password to protect the saved file.

We're looking for "CN=nam-idp.lab.ba" in the example. If you click on the certificate you can see in the certificate details that the serial number also matches the serial number in the server "hello packet."

6. Export the Private/Public Keypair and type a password.

7. Save the file as e:\trace\nam-idp_lab_ba_exportedCert.pfx

Click to view.

Figure 3

Figure 3 - Exporting the certificate file

Click to view.

Figure 4

Figure 4 - Certificate information

Click to view.

Figure 5

Figure 5 - Exporting the Private/Public Keypair

Because the exported file is encrypted with a password, Wireshark will not be able to decrypt this file. Therefore, this file now needs to be converted to a file readable by Wireshark. This can be done with OpenSSL. The syntax is described in the example below.

Go to the directory where the saved .pfx file is stored and use OpenSSL to convert the file to a non-encrypted file. The "pkcs12" parameter tells OpenSSL we're dealing with a .pfx file containing both public and private key. The "in" and "out" parameters are self-explanatory. The "nodes" (read no-DES) parameter tells OpenSSL to save the file without encryption. OpenSSL will ask for the password you provided during the export. Be careful with the output file, as it contains the server's private key in clear text!

Click to view.

Figure 6

Figure 6 - OpenSSL parameters

Configuring Wireshark for SSL Decryption

Now we have everything needed to configure Wireshark for decrypting the SSL data. To set this up,

1. In Wireshark, go to Edit > Preferences.

2. In the Preferences dialog, select SSL in the Protocols sections. The two first fields that will reassemble data should be enabled to make the data easier to read.

3. In the "SSL debug file" field, specify a filename where debug data can be written to. Depending on the size of the trace this file can become very big.

4. In the "RSA keys list" field, make sure all the necessary information is specified to find the packets, extract the needed encryption keys, and decrypt all the packets. The correct format is:

<server_ip_address,ssl_port,protocol_used,path_to_cert_file>

After all this is specified, Wireshark will search for the specified SSL packets and decrypt the application data.

5. Fill the "RSA keys list" field with the data gathered earlier:

• Src IP address
• Src TCP port
• Protocol is http
• Key file we decrypted with OpenSSL

The resulting field will be:

149.44.38.230,8443,http,e:\trace\nam-idp_lab_ba.pem

6. Specify a debug file to troubleshoot the plug-in in case it doesn't work.

7. Click Apply and OK.

The packets named "application data" have become typical http packets, so now you can see and analyze the http data.

Click to view.

Figure 7

Figure 7 - Wireshark SSL Preferences

Click to view.

Figure 8

Figure 8 - HTTP preference info

Important Note for Firefox Users

You might have to change the Cipher Suites in the Firefox browser, as the Wireshark decryption plug-in only works with RSA key exchange.

To check which suite is used, look again into the server "hello packet." If the Cipher Suite starts with TLS_RSA_*** you should not have any problems. If this Cipher Suite starts with TLS_DHE_*** you should (temporarily) change your browser settings. In Firefox you can do this by going to URL, "about:config", Filter for "security.ssl3.dhe" and set the values to "false". If you try again, you should be able to decode.

Click to view.

Figure 9

Figure 9 - Alternate Firefox setting

Disclaimer: As with everything else at Cool Solutions, this content is definitely not supported by Novell (so don't even think of calling Support if you try something and it blows up).

It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.