EVGA Killer Xeno Pro: The Impact of Network Offloading
by Derek Wilson on July 3, 2009 4:20 AM EST- Posted in
- Networking
Mostly Deterministic Testing
Designing tests to determine the real world benefit of the Killer Xeno Pro has proven quite difficult. Even though frame rate testing with single player games isn't strictly deterministic, proper tests can produce results that are fairly consistent and have low variance. We haven't included many MMOs or multiplayer games that don't utilize timedemo functionality in our graphics hardware tests specifically because they are very hard to appropriately benchmark. We can get ideas about performance from play testing, but graphs and charts have a certain finality and authority to them that we just don't want to lend to tests that we can't stand behind are representative of relative performance.
We did come up with one test that is highly reliable, however. This test is a side by side comparison of framerate when playing EVE online. We ran two different computers side by side with exactly the same hardware and software setup except that we installed the Killer Xeno Pro in one box. Both instances of EVE undocked characters in Jita (a system that typically hosts about 1000 players at a time) and flew to nearly the same spot. Because EVE allows players to choose something to "look at" and centers the camera on that object, were were able to have two instances of the game running with players very near each other (requiring very similar network data) and with exactly the same graphical load (because they were looking at the same thing).
Our EVE test is in a place where there were a very high number of other players and we were able to eliminate as many other factors as possible from testing. This test showed no difference in performance with or without the Killer Xeno Pro:
| EVE Test | Average FPS | Min FPS | Max FPS |
| Killer Xeno Pro | 84.3 | 67 | 99 |
| On-board NIC | 84.5 | 68 | 98 |
We attempted testing in other multiplayer environments like Team Fortress 2 and World of Warcraft, but we couldn't eliminate graphics as a factor when side by side testing with different players like we could in EVE. If we did sequential testing, one run to the next had very high variability even on the same hardware (due to the influence of other players).
We did run some tests in not very highly populated areas of WoW and found that framerate and ping seemed to show no difference. This might be different for highly populated areas, but again we couldn't be very deterministic in testing this.
In trying to do the similar testing with Team Fortress 2, the Killer Xeno Pro would be faster in once instance and slower in the next. There was no real consistency to our data in this case.
Bigfoot claims that there is benefit from the hardware in games like WoW, Team Fortress 2, Counter Strike: Source, and other games with high volumes of network traffic. We really do not doubt the capability of the hardware to provide some sort of difference, but our tests just are not deterministic enough to appropriately compare the hardware. But in a way this does tell us something very important: factors other than client side networking (like the performance of the network itself, other players, servers, and potentially graphics) have a much higher impact on performance.
The Killer Xeno Pro does suggest another advantage: bandwidth prioritization and throttling. The hardware is capable of Quality of Service (QoS) like prioritization on a per application basis, and every application can have upload and download bandwidth caps. This could potentially help out when multiple network heavy applications are vying for bandwidth. We decided to test this with both EVE (for framerate and download speed) and WoW (for framerate and latency).
In our EVE test, we used uTorrent to download a 650 MB file while we played EVE. Because we had to do this test sequentially rather than side by side (the bandwidth demand from on torrenting computer would negatively impact the bandwidth available to both PCs -- a point we'll come back to later), our frame rates aren't directly comparable because of all the other player activity. Please keep in mind that fluctuations in the multiplayer environment make this a non-deterministic test despite the fact that framerates are similar.
| EVE Test + Torrent | Average FPS |
| Control (no download) | 98.7 |
| Killer Xeno Pro w/ Prioritization | 98.4 |
| On-board NIC | 98.5 |
We did, however, see a very large difference in the time it took to download our torrent.
| Torrent Time + EVE Test | Download Time in Minutes |
| Control (no game) | 27 Minutes |
| Killer Xeno Pro w/ Prioritization | 69 Minutes |
| On-board NIC | 30 Minutes |
Since we can't get an assessment of ping times in EVE, we did some testing on WoW in the same unpopulated area. Normalized to the average latency we experienced while not downloading a torrent, here's the latency incurred by downloading a torrent:
| WoW Test + Torrent | Increase in average Latency |
| Killer Xeno Pro w/ Prioritization | 15ms |
| On-board NIC | 25ms |
Even with these latency differences, our framerates were very constant at about 54 FPS with 0.4% difference between the three different setups.
Again, this might have a larger impact in a more highly populated area in WoW. But the hardware does show a ping time advantage over our on-board NIC when downloading a torrent while gaming.
Facebook
Twitter
RSS
121 Comments
View All Comments
mindless1 - Saturday, July 4, 2009 - link
The thing is, even with a slower CPU and PCI bottleneck, the network processing still isn't a substantial % of processing by the CPU, and the traffic for gaming not bottlenecked by PCI bus.Even a lowly Celeron 500MHz isn't much of an issue if jumbo frames are used, though CPU still has to be seen as a bottleneck to the gaming itself.
has407 - Sunday, July 5, 2009 - link
Jumbo frames won't do squat in this case, and will likely cause worse problems. Even in well--managed and closed environment, expect very little gain unless you're using a very fast SAN, fast switches, and a network admin who knows what they're doing.Do the math: even for 1Gbe networks the efficiency gain for most apps using jumbo frames is noise. For 10Gbe you might notice it if you've got enough CPU and IO bandwidth; for the typical home network, it's not worth considering.
davecason - Saturday, July 4, 2009 - link
PCIe, not PCI.Theunis - Saturday, July 4, 2009 - link
I wonder if it would be possible to use this board with my Linux x86_64 machine. LOLWouldn't it be cool to run specific applications compiled for PPC, to run on this board? Does it come with it's own RAM?
ShawnD1 - Saturday, July 4, 2009 - link
If they're trying to market this thing as something to reduce CPU usage, it doesn't really make much sense to test it with the fastest processor you can find. Try it with a CPU that has no speed at all, maybe a celeron or sempron.Of course that's not a real world test, but are any tests on Anandtech realistic? I don't run my games at 3000x2000 resolution, but ridiculous tests like that show us what a video card can do. For CPU tests we're looking at Phenom II and Core 2 Quad systems running games at 800x600 and getting 200fps. It's a ridiculous test, but it isolates the hardware being tested.
The methodology in the article, in my opinion, is like testing a bunch of video cards at 800x600. Seeing that every video card is getting 200fps (the CPU bottleneck), the conclusion would be that upgrading the video card is a waste of money. Similarly, testing something that reduces CPU bottlenecking should not be done with a CPU that isn't bottlenecked by any game in existence.
Gannon - Saturday, July 4, 2009 - link
This product has no real market, it's just an excuse to charge more money to clueless among the gaming population.What they should really do with this card is make it multi port and a router, I would love to ditch my piece of shit router that requires constant reboots because of someones Wifi dropping (it works fine for wires connections). If they could build a wireless router network add in card + network stack offloading + opening up the card to developers, then I'm in. Screw the "gaming" portion of it, how bout building a quality product gamers would want?
Such as: Bandwidth control (so users can't flog the connection can have their bandwidth limited so it doesn't fsk_up your ping in an online twitch game like quake 3, etc, etc... othe routers have attempted this like Dlinks "Gamefuel", so one can have torrents + game at the same time, no one has really done it very though.
The networking stack is the least of a gamers worries on a modern computer, there is a reason everything has become more integrated over time (audio + network), with the rise of the internet NOT having an ethernet port on a computer is stupid and most onboard NIC's are so good now-a-days unless you are doing some serious file transferring you don't need it.
Anyone claiming to see a performance benefit is shitting you, the real problem lies in input output latencies to devices, RAM and hard drives.
I'll take audio + NIC integrated on future CPU's with an integrated memory controller over add-in shite that is just going to fade away over the next 5-10 years.
Gannon - Saturday, July 4, 2009 - link
Also the next real major speed up for games is in Solid state disks, what we really need is:-Newer faster Memory technology, CPU spends most of it's time waiting on RAM (and ram is damn fast comparable to hard drives and even solid state drives).
-Newer faster permanent storage (SSDa and beyond)
When solid state disks mature and they finally come out with a chipset that can really take full advantage of SSD's and the bandwidth they offer we'll see a lot more performance improvement.
Try playing a game that chugs on an old hard drive, then put it on an SSD, notice it's not as choppy when things get harry. I noticed this when I moved many of my games from an older 320GB drive to my 1TB drive, games that were slow/choppy suddenly got a speed boost because the drive IO was a sever bottleneck, I can't wait to get an SSD once their capacity expands and price comes down to saner levels.
navilor - Saturday, July 4, 2009 - link
I have on my old machine (a Core2Duo E6600) the Bigfoot KillerNIC M1. I thought that it was a complete crock of [expletive deleted] until a buddy of mine bought one and reported a much better gaming experience.And what exactly did it do?
He played Everquest and it lowered his latency a lot. This means that his character is a lot more responsive to what is happening around him. Yes, you can disable Nagel's algorithm and do something similar, but that wasn't the end of it.
He was able to report being able to run with higher graphical settings enabled. It also removed a stutter that he previously did not notice.
So I sucked it up and decided to blow some cash on that product.
Excellent investment. No longer did my CPU have to manage TCP/IP packets. UDP packets were invisible (via WireShark) yet passed through to the OS without issue (there is a Game setting and an Application setting for those who need it).
This network card offloaded work so my CPU could be used to manage things. Now you might think that my CPU was lame. World of Warcraft, which is the game I play, barely touched the CPU at all. No matter how powerful your CPU is it still has to deal with networking.
Unless it doesn't because you have a KillerNIC.
Now did it lower my latency? No, but it did for my friend. Did disabling Nagel's algorithm help? Yes, but it didn't smooth out my frame rate. Combine the two and you will notice a difference.
On my new rig (Core i7 920 with a GTX 295 and 12GB of RAM) disabling Nagle did jack for me. I am considering either purchasing the new Xeno Pro or possibly stripping the M1 out of my old system. My card can run a firewall on it (iptables) so I don't have to burden Windows with that overhead.
Oh, and high end servers run network cards that do TCP offloading. I'm vaguely certain that those types of cards are there for a reason.
You can prioritize your packets all you want on the network, too. That is always a good first step. What the network cannot do is reduce the time it takes for your application to:
1). Generate a packet
2). Have it go through the windows networking stack
3). Go through the Windows network driver
4). And then out the cable.
The Killer products remove step two and the overhead of step three.
Windows, by default, likes to lump packets together for high speed transmission. See also Nagle's Algorithm. That can be disabled via a few registry hacks. Removing the overhead of Windows compressing those and shoving them through the driver smooths things out. If you have a KillerNIC then you can still manually disable Nagel (which lightens the workload of packet management a small amount) and let the KillerNIC worry about the rest.
So what this means is that the Killer products prioritize the packets INSIDE of your machine BEFORE they hit the home network to then be prioritized on your internet facing router.
DerekWilson - Sunday, July 5, 2009 - link
I get the things that the Killer is doing and that those things are real ...But if your friend took the $120 for the Killer Xeno Pro (or the likely much higher cost of the M1 at the time) and spent it on a faster CPU, the benefits would have extended to much more than just making packet processing faster.
It would have benefited many other applications as well in addition to delivering the performance needed for smooth network play.
has407 - Saturday, July 4, 2009 - link
I won't argue with your experience, as I haven't used one of these NICs. However, is it a cost-effective or appropriate way to solve the problem? Color me skeptical; the evidence is at best inconclusive.1. Nagle applies to TCP, not UDP.
= You aren't going to see any improvement disabling Nagle for apps that use UDP.
2. TPC_NODELAY is a way for apps to bypass Nagle.
= Apps with time-sensitive needs and that use TCP should use it. Nothing you can do about this, but I would hope and expect game developers would be cognizant of it and use it appropriately (or use UDP).
3. An old 2.2GHz Core-2 can drive > 150KBs of 1-byte packets with TCP_NODELAY; > 500KBs using 1-byte UDP packets; > 225MBs with 16KB packets; and peaks at ~300K segments/sec.
= For modern CPU's, CPU time is noise given typical Internet bandwidth.
= Latency/CPU due to the network stack is noise.
4. Some NICs have features which you may want to disable. E.g.:
- Interrupt coalescing. This reduces CPU load by not generating an interrupt for every packet. That may be counterproductive for games.
- Large send offload. Removes the CPU overhead of segmenting large packets into smaller ones and moves it to the NIC. Doubtful there'd be much difference unless the game is sending large packets (which I expect isn't the case).
- Jumbo frames. Don't. In this scenario they're at best a NOP and at worst will degrade performance.
In short, will the Killer NIC perform any better than a properly tuned system? I doubt it. Is the $premium worth the equivalent amount spent on a faster CPU or GPU, or the time required to tune the system? Your call, but again I doubt it.
p.s. No, disabling Nagle does not reduce "the workload of packet management a small amount". It exists to reduce per-packet overhead by coalescing small messages into larger packets.