TCP Performance Inside an ESP Tunnel

This problem took a little more time to analyze as the users tcp session was inside an ESP tunnel, from the Office to the Data Center. I didn’t have much experience with ESP but with some work was able to develop a spreadsheet to visualize the problem. Of course I did this using PowerQuery and PowerPivot.

The first problem was reordering of the ESP packets. I knew we were getting reordering as the Checkpoint Firewall was often dropping packets due to “replay attack” detection.  It seems the replay window on the FW was 64, that is, a packet can be late or early by 64 before a replay attack is detected. We fixed this by white listing the ESP traffic in the IPS. That fixed the replay attack messages but the file transfer speed did not improve.  I needed a way confirm packet reordering was not still occurring so I developed these charts.

The top chart is sum of ip length / second over time of  day. The bottom chart is reordering count (early and late) per second over time of day.


Using this I could see reordering was still occurring but to a lesser degree. Another call the IPS vendor, another white list of the ESP traffic but this time at layer 2 (previous was just in the inspection engine) and the reordering was gone and TCP performance shot up to 30 MBytes/s. All good but occasionally we the file transfer speed would drop from 30 down to 3 again. In this case we were getting packet loss of the ESP packets.

Now the bottom chart shows ESP packet loss. Each step is the number of packets lost in a given second. This is basically gaps in the ESP sequence numbers. Top is sum of ip length/s.


Here are the steps to get your own chart.

  1. Get a packet capture of a file transfer. You are capturing the ESP packets.
  2. Convert the capture to CSV: tshark.exe” -r <fileName>.cap  -T fields  -E separator=, -e frame.number -e frame.time -e ip.src -e ip.dst -e ip.proto -e ip.len -e esp.spi -e esp.sequence >  <fileName>.csv
  3. Import into Excel using PowerQuery (PQ) and parse the fields.
  4. Push the parsed CSV in to PowerPivot (PP) using PQ. Now you can add a Pivot Table from PP, on rows put esp.spi and SSum Values is Count of esp.spi. The largest esp.spi will most likely be you file copy.
  5. Now go back and filter you capture to the big esp.spi, save, convert to CSV, this is data for the charts. Notice that esp.spi is only used in one direction, you want the from sender direction.
  6. Load this CSV into PQ, parse out the fields. Also convert the time to Hours, Minutes and Seconds. You frames should start at one increasing in order. This reflects the order the frame was received. Do a little math on the esp sequence number so it starts a 1 too. Now we have frame # and the esp seq # aligned. Push this into PP and plot the max and min of esp sequence on a Pivot Chart and you are done.

I hope you find this useful.







Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s