EVGA Killer Xeno Pro: The Impact of Network Offloading
by Derek Wilson on July 3, 2009 4:20 AM EST- Posted in
- Networking
Mostly Deterministic Testing
Designing tests to determine the real world benefit of the Killer Xeno Pro has proven quite difficult. Even though frame rate testing with single player games isn't strictly deterministic, proper tests can produce results that are fairly consistent and have low variance. We haven't included many MMOs or multiplayer games that don't utilize timedemo functionality in our graphics hardware tests specifically because they are very hard to appropriately benchmark. We can get ideas about performance from play testing, but graphs and charts have a certain finality and authority to them that we just don't want to lend to tests that we can't stand behind are representative of relative performance.
We did come up with one test that is highly reliable, however. This test is a side by side comparison of framerate when playing EVE online. We ran two different computers side by side with exactly the same hardware and software setup except that we installed the Killer Xeno Pro in one box. Both instances of EVE undocked characters in Jita (a system that typically hosts about 1000 players at a time) and flew to nearly the same spot. Because EVE allows players to choose something to "look at" and centers the camera on that object, were were able to have two instances of the game running with players very near each other (requiring very similar network data) and with exactly the same graphical load (because they were looking at the same thing).
Our EVE test is in a place where there were a very high number of other players and we were able to eliminate as many other factors as possible from testing. This test showed no difference in performance with or without the Killer Xeno Pro:
| EVE Test | Average FPS | Min FPS | Max FPS |
| Killer Xeno Pro | 84.3 | 67 | 99 |
| On-board NIC | 84.5 | 68 | 98 |
We attempted testing in other multiplayer environments like Team Fortress 2 and World of Warcraft, but we couldn't eliminate graphics as a factor when side by side testing with different players like we could in EVE. If we did sequential testing, one run to the next had very high variability even on the same hardware (due to the influence of other players).
We did run some tests in not very highly populated areas of WoW and found that framerate and ping seemed to show no difference. This might be different for highly populated areas, but again we couldn't be very deterministic in testing this.
In trying to do the similar testing with Team Fortress 2, the Killer Xeno Pro would be faster in once instance and slower in the next. There was no real consistency to our data in this case.
Bigfoot claims that there is benefit from the hardware in games like WoW, Team Fortress 2, Counter Strike: Source, and other games with high volumes of network traffic. We really do not doubt the capability of the hardware to provide some sort of difference, but our tests just are not deterministic enough to appropriately compare the hardware. But in a way this does tell us something very important: factors other than client side networking (like the performance of the network itself, other players, servers, and potentially graphics) have a much higher impact on performance.
The Killer Xeno Pro does suggest another advantage: bandwidth prioritization and throttling. The hardware is capable of Quality of Service (QoS) like prioritization on a per application basis, and every application can have upload and download bandwidth caps. This could potentially help out when multiple network heavy applications are vying for bandwidth. We decided to test this with both EVE (for framerate and download speed) and WoW (for framerate and latency).
In our EVE test, we used uTorrent to download a 650 MB file while we played EVE. Because we had to do this test sequentially rather than side by side (the bandwidth demand from on torrenting computer would negatively impact the bandwidth available to both PCs -- a point we'll come back to later), our frame rates aren't directly comparable because of all the other player activity. Please keep in mind that fluctuations in the multiplayer environment make this a non-deterministic test despite the fact that framerates are similar.
| EVE Test + Torrent | Average FPS |
| Control (no download) | 98.7 |
| Killer Xeno Pro w/ Prioritization | 98.4 |
| On-board NIC | 98.5 |
We did, however, see a very large difference in the time it took to download our torrent.
| Torrent Time + EVE Test | Download Time in Minutes |
| Control (no game) | 27 Minutes |
| Killer Xeno Pro w/ Prioritization | 69 Minutes |
| On-board NIC | 30 Minutes |
Since we can't get an assessment of ping times in EVE, we did some testing on WoW in the same unpopulated area. Normalized to the average latency we experienced while not downloading a torrent, here's the latency incurred by downloading a torrent:
| WoW Test + Torrent | Increase in average Latency |
| Killer Xeno Pro w/ Prioritization | 15ms |
| On-board NIC | 25ms |
Even with these latency differences, our framerates were very constant at about 54 FPS with 0.4% difference between the three different setups.
Again, this might have a larger impact in a more highly populated area in WoW. But the hardware does show a ping time advantage over our on-board NIC when downloading a torrent while gaming.
Facebook
Twitter
RSS
121 Comments
View All Comments
MrHorizontal - Friday, July 3, 2009 - link
Could you pit the Bigfoot NIC's against some other NICs.I'd like to know what the difference between using a 'Server Adapter' such as the Intel PRO/1000 PT Dual port would be (on a LACP compatible switch, even a cheapo one like the Netgear GS108T) for example, or even a direct comparison with the single port Intel PT card, since they all do a lot of TCP offloading work, though not the entire stack like the Bigfoot seemingly does...
I do give a LOT of attention to my network connection being a very keen MMORPGer, so I'd genuinely like to know what I can do end-to-end beyond just having a good ISP to make my network as good as it can be with a fairly reasonable eye on cost (ie not forking out for f5 and Cisco kit!)
DerekWilson - Friday, July 3, 2009 - link
The Killer isn't targeted as competition to server NICs, some of which do a lot more like 10GbE and have specific optimization for handling massive numbers of VMs all trying to use the network at once ... Even server NICs like the Intel ones you point out -- even if they did do full network stack offloading -- are very likely not going to be of any more benefit to gamers than the Killer Xeno Pro.But there is no replacement for testing.
It is something we will keep in mind, but with the limited usefulness the Killer Xeno Pro shows it's sort of hard to justify putting a lot more time and energy into this sort of investigation. I'd like to satisfy my own curiosity on the subject, so maybe it will be something we get up ... but no promises. :-)
MrHorizontal - Monday, July 6, 2009 - link
The Killer may not be targetted to server NICs, but as I and a lot of the other comments have noted the price point of the Killer is equal to that of a server adapter. As for whether or not the stack offloading in the Killer NIC and the TCP/IP Offload Engine in the Server Adapters are different and perform differently, only benchmarking both cards would actually provide a good and honest result.I'd like to see ping tests and iperf results using both cards, with and without jumbo frames, using Link aggregation where possible (LACP is effectively SLI for a network card after all) and a comparison with the best and worst mobo adapters versus the Intel / Netgear 'desktop' NICs and the Intel server NIC's versus the Killer NIC.
Essentially what I'm asking from Anandtech is to find what kit we need to get and what configuration changes we need to make to the registry, routers and switches to get absolutely rocking performance from our LANs... which are after all one thing we can control and where a significant amount of lag originates over and above the latency in the ISP's network.
That would be a useful article of the quality that I'd expect from Anandtech. Harping on about a product that noone including yourselves really believes has a market isn't a good report...
UnclePauly - Friday, July 3, 2009 - link
WOW! COUNT ME IN FOR TWO!Myg - Friday, July 3, 2009 - link
This article started with the best of intentions, but fell flat on its face when it came to real value (just like the card apparently).You can't expect a network card to increase FPS (we all know thats just a marketing ploy) and you lot know better and should be able to see through that.
This is a networking device, so it should be treated like one. A suite of dedicated server programs should of been used for the testing. It is terribly lazy of anandtech and seemingly a growing trend to not bother with going that extra mile (which made you guys popular in the first place).
mesiah - Saturday, July 4, 2009 - link
They are testing it for what its marketed towards. The manufacturer makes clear claims in their marketing and this review tested those claims. While it may have some value as a server card, that is not what it was designed for and marketed for. Corvettes have a shitload of power, but whens the last time motor trend talked up their towing capacity? Its beyond the scope of the article.HotdogIT - Friday, July 3, 2009 - link
Why would they test it as a "networking device", when it's clearly being markteted to gamers? The manufacturer themselves claim ping improvements and FPS improvements in GAMES, so not testing GAMES is silly.Every site who has tested this product, HardOCP, Anandtech, TomsHardware, have come to the same conclusion: It's a 125$ NIC, that's better spent on other components. It either has no impact in gaming, or so minimal that it's within the realm of just dumb luck.
The only benefit it might have is throttling of network connections, be it torrents (which you can throttle manually, unless you're an idiot) or downloads (Firefox has a plugin to throttle, and I'm sure it can be done with IE, somehow).
hooflung - Friday, July 3, 2009 - link
In fact, it doesn't do TCP/IP offloading the way you think. It does UDP offloading. That is why your EVE test is flawed. EVE uses a TCP not UDP because it needs a guaranteed connection. The Xeno and original Killer claim WoW FPS goes up, specifically in Really overworked zones, because it offloads that UDP to the NPU.So thanks for your totally crap review filled with misinformation.
DerekWilson - Friday, July 3, 2009 - link
Actually, bigfoot makes it very clear that it's the whole network stack that is offloaded. this includes TCP and UDP. which is also why it has a separate mode for applications that expect a dedicated software network stack.In fact, the WoW example is important, as they told us that WoW uses TCP and NOT UDP -- thus it was not specifically detected as a "game" application (it's the "unknown" blip in the bandwidth control pages).
The Killer Xeno is definitely able to offload TCP because, as we showed, it reduced WoW latency (and potentially could benefit overall performance in heavily populated areas).
I'm not sure whether EVE uses TCP or UDP, but it doesn't matter -- the Killer Xeno Pro should handle both just fine.
hooflung - Friday, July 3, 2009 - link
Funny that, since I have played EVE Online for nearly 6 years and also have had a killer NIC since they came out.I have talked to their engineers and they told me that the Driver does not do TCP accelleration unlike UDP. Also, WoW uses UDP unless the WoW developers are wrong, which I doubt, and the Bigfoot engineer I talked to, aka the guy who designed it who also put a few hundred million into Intel's pocket by creating the offload engine for the Intel Server NIC's is wrong.
I'll just assume you got your facts mixed up. Also, EVE absolutely uses TCP. If you for a single moment loose your TCP connection you will disconnect. You don't from WoW... because they use UDP after you handshake and authenticate to their login server.