802.11n and the truth about throughput

I recently purchased a shiny new Trendnet TEW-639 to replace an aging 802.11g Netgear router. I decided to go with an 802.11n draft 2.0 router since they are widely available and not much more expensive than an 802.11g. To go along with it, I also bought a shiny new DLink DWA-652 Xtreme N notebook adapter.

Before I purchased my new hardware, I wanted to test out my old 802.11g hardware to see what my transfer rates were so I could compare them to the new hardware. My test methodology is quite simple. Transfer a 200MB file via ssh (scp) from my Desktop (core2 Duo, 100 Mbps Ethernet) to my laptop (IBM Thinkpad T42, 1.7Ghz Pentium-M, 802.11G built-in wifi ). I tested the transmission speed in 2 places. 1) about 10 feet away from the router, in the same room, with no interfering electronics in between. 2) In my bedroom which is separated form the router by about 15ft (assuming you can walk through walls) and 2 walls. I was quite surprised when I saw that my average transfer speed (in both places) was about 2.0MBps instead of something a little closer to the theoretical 6.0MBps that should be possible with a 56Mbps connection.

I did some research on 802.11n, chose a decent router, then found a decent PCMCIA adapter that was Linux compatible (D-Link DWA-652 is based on the new Atheros chipset and used the ath9k driver) and set up for testing. I placed my order and patiently waited. A few days later a package was on my doorstep when I came home from work. :)

I hooked up the new router and configured it mostly with default settings. I verified that it was working correctly and then I turned my attention to the new adapter! I put it in my laptop and disabled the on-board Wifi card in the bios. I booted Ubuntu 8.10, which supports the new card out of the box) and proceeded to do my SSH test. I quickly noticed that something was terribly wrong... I could not hold a connection for longer than 30s!! Ok... what to doooo.. AHH.. I should apply the kernel update thats been sitting in my updater for 2 months! So a quick update and 1 reboot later I can hold a connection for about 2-3 minutes. After doing some more research I stumble upon a page at wireless.kernel.org which gives instructions for obtaining a fully standalone version of the Linux Kernel's most up to date wifi stack and drivers. I noticed my driver (ath9k) had been updated the day before. So I downloaded, compiled and installed the latest and greatest drivers and my wifi card happily holds a connection for as long as i want now. K... on to the benchmarks! When I repeated the same tests I was getting about 3.0MBps on average. This is a 150% boost! This seems ok until you remember that 802.11n advertises a 300Mbps theoretical maximum!! If I'm upgrading from 54mbps to 300mbps I might expect to get roughly a 600% boost in performance. So after trying a bunch of stuff and repeating my benchmarking for several hours heres what I found.

ANTENNA POSITION MATTERS
With all three of my antennas pointing straight up performance sucks. With 802.11n's MIMO technology taking advantage of multiple antennas simultaneously you want to make sure your antennas are NOT all pointing in the same direction. I obtained a significant increase in throughput simply by bending two of my 3 antennas at a 45 degree angle (relative to the "standing straight up" position) in opposite directions.

Router defaults SUCK!
So 802.11n can do the 802.11a/g thing and only use a 20Mhz chunk of bandwidth (1 channel) or it can practically double throughput and use a 40Mhz chunk (2 channels). By default my router was set to only use 20Mhz in order to reduce interference with other wireless networks. After Switching my router to allow 40Mhz mode things improved quite a bit.

Wifi Routers are like flying cats!
The higher off the ground they are the happier they are! Get the router off of your desk! Put it on something tall, or mount it to the wall. If you must have it on a desk then make sure to stand it up so the antennas are as high as possible.

SNR matters!
Learn how to properly read your SNR. iwconfig output usually shows you two numbers:

Signal level=-30 dBm
Noise level=-87 dBm

The signal level is negative and it tells you how much your signal is dropping off as it is transfered from the router to your wifi card. THE LOWER THE BETTER!! And by lower I mean the absolute value! So a -30 is better than a -50 which is better than a -60! I find that a value of -30 generally gives me 100% signal strength.

The noise level (also a negative number) works in the opposite direction. The higher the absolute value of this number the less your signal is being affected by noise. So a -90 means less noise and a better connection then a -80 which is better than a -60. I find that -85 or higher leads to a solid signal at 100% strength.

Results??
Ok, so after all of this tweaking I was able to get an actual maximum throughput of about 7.2 MBps (57.6Mbps). On average, I get 5.5 MBps (44MBps) which is way lower than the advertised 300Mbps but it is still about 3x my original performance which is good.

HTTP and Windows comparison
I wanted to test the Windows driver to see what kind of performance I could get with a driver written by the manufactuer. But Windows SSH (WinSCP) performance sucks BIG TIME as everyone knows by now... I could only get 2.5MBps out of it. So I decided to switch to an HTTP workload for the rest of my benchmarks.

Using Windows XP and the latest driver for this card gets me about 9.5 MBps(76Mbps) max and 9.0MBps(72Mbps) average when transferring the 200MB test file via HTTP from a local Apache2 server using wget. The same transfer using wget on Linux gets me 9.0 MBps(72Mbps) max and 7.0 MBps(56Mbps). Not too shabby ;), but still not as good as the native driver supported by the manufacturer who has all of the specs and knows exactly how the hardware works...

One thing I'd like to mention is that the Linux benchmarks seemed to fluctuate rather wildly (unlike Windows. One moment I could be seeing a transfer rate of 9MBps, then it could drop to 4MBps and then slowly climb back up to 9. Also, it seems that not all TCPIP sessions/connections are created equal. With both the SSH and HTTP workloads, I could make a connection, transfer the file and get an average rate of... say... 4MBps. If I immediately repeat the exact same test I could see a much higher transfer rate! over double in some cases. I'm not sure what to blame this oddness on.

Comparing 9.0MBps to 300Mbps
So 802.11n advertises 300MBps. This is really a terrible case of false advertisement! Here is my reasoning.

1) From my research it seems to me that the 300Mbps figure is assuming 150Mbps per direction (send/receive). If this is true (I would LOVE for someone to confirm this!) it means that 150MBps is our new theoretical maximum because most (ALL??!?!?) high throughput network based workloads require the sending of a large amount of data in a single direction NOT in both directions simultaneously. Examples: file transfer, streaming video or audio. Bidirectional workloads tend to need very low throughout... instant messaging, IRC, TCP control (ACK's in response to packets received), HTTP requests, etc.

2) The overhead for the actual wireless connection at the PHY layer is documented at about 25% (I lost my citation for this). This means you only have 75% of the theoretical maximum.
So 150Mbps*0.75 = 112Mbps.

What is this number in MBps? 112/8 = 14MBps.

Now.. 14MBps is our new THEORETICAL MAXIMUM! This is not accounting for overhead introduced by the hardware, driver, OS, and transport/applications layers(TCP,SSH,HTTP).

On a 10OMbps (12MBps) network I obtain an average throughout of 10MBps via my HTTP workload. This means that, taking real world overhead into account, I get about 83% of the advertised rate.

With 802.11n my theoretical max is 14MBps and I get 9.0MBps This is about 64% which a rather large difference. remember, this is AFTER subtracting 25% overhead for the PHY layer.

Sure 802.11n is faster than the other network technologies :) HOWEVER, I call shenanigans in regards to the IEEE claim of 300Mbps!

In conclusion, I'd like to point out that I did manage to improve the peformance of my original wireless network by about %400-%450 percent. So it isnt all bad :)

Questions? Comments? Can you support or refute any of my claims or testing methodologies? I'd love to hear about others experiences with 802.11n!!!!!! How fast can you get it to go? What can we do to make things better? :-)