A nice feature of the VMXNet3 vNIC in vSphere is Receive Side Scaling.
For detailed info, check out http://technet.microsoft.com/en-us/network/dd277646.
I’ll try to summarize it here:
Without RSS, all incoming network traffic is handled by CPU 0. This could cause a bottleneck on high Network IO machines resulting in an overloaded CPU 0 (where CPU 1, 2, 3, … might be idle). With RSS enabled, incoming network traffic is handled by all available CPUs.
RSS is available inside a VM on vSphere. But, you’ll need to have the VMXNet3 adapter in your VM.
Let me show you how to enable this on a Windows 2008 R2 VM.
For starters, make sure your VM runs on Hardware Version 7. Second, you’ll need more than one vCPU in the VM. After all, it would be useless to enable RSS on a single vCPU machine.
Next, verify that the vNIC is of type VMXNET3. If it is not, remove the current vNIC (make a note of the IP address) and add a new one of type VMXNET3.
That’s it for the configuration on the vSphere side. Now we’ll have to enable it inside the VM. We’ll use a Windows 2008 R2 machine for this, but know that RSS is available on 2003 SP2 and 2008 as well.
By default, RSS is enabled on Windows 2008 R2. This can be checked by running the following command:
netsh int tcp show global
You’ll notice Receive-Side Scaling State is enabled.
Allright, so let’s test this!
To test RSS, i have 2 VM’s running iPerf. One is the client (sender) and the other is the server (receiver). I start iPerf with 16 concurrent threads to generate lots of network traffic. If all goes well, we should see RSS in action on the iPerf server.
I’ll fire up perfmon.exe and monitor the % Processor Time of CPU 0 and 1. When iPerf kicks in and receive network packets, we immediately see CPU activity:
Whoops, looks like CPU 0 (the red one) is doing most of the work and CPU 1 (the green one) isn’t allowed to play along…
So RSS is enabled on the Operating System, but it looks like it’s not functioning. The fact is thatRSS must be enabled on the vNIC inside the VM as well! For that, open up Device Manager and open the Advanced Properties of the vmxnet3 Ethernet Adapter.
Look for the RSS property and set it to Enabled (it is Disabled by default).
Let’s rerun the iPerf test and check CPU 0 and CPU 1 in perfmon:
That’s more like it! All CPUs are playing along now and we have just removed a potential bottleneck!
Bottom line:
– RSS is available on the VMXNET3 vNIC adapter
– RSS is enabled by default on Windows 2008 R2
– You have to explicitly enable RSS in Device Manager on the VMXNET3 adapter.
It’s up to you to enable RSS, but personally, i would enable it if you have high amounts of incoming network traffic and you see an overloaded CPU 0.
