EVGA Killer Xeno Pro: The Impact of Network Offloading
by Derek Wilson on July 3, 2009 4:20 AM EST- Posted in
- Networking
Mostly Deterministic Testing
Designing tests to determine the real world benefit of the Killer Xeno Pro has proven quite difficult. Even though frame rate testing with single player games isn't strictly deterministic, proper tests can produce results that are fairly consistent and have low variance. We haven't included many MMOs or multiplayer games that don't utilize timedemo functionality in our graphics hardware tests specifically because they are very hard to appropriately benchmark. We can get ideas about performance from play testing, but graphs and charts have a certain finality and authority to them that we just don't want to lend to tests that we can't stand behind are representative of relative performance.
We did come up with one test that is highly reliable, however. This test is a side by side comparison of framerate when playing EVE online. We ran two different computers side by side with exactly the same hardware and software setup except that we installed the Killer Xeno Pro in one box. Both instances of EVE undocked characters in Jita (a system that typically hosts about 1000 players at a time) and flew to nearly the same spot. Because EVE allows players to choose something to "look at" and centers the camera on that object, were were able to have two instances of the game running with players very near each other (requiring very similar network data) and with exactly the same graphical load (because they were looking at the same thing).
Our EVE test is in a place where there were a very high number of other players and we were able to eliminate as many other factors as possible from testing. This test showed no difference in performance with or without the Killer Xeno Pro:
| EVE Test | Average FPS | Min FPS | Max FPS |
| Killer Xeno Pro | 84.3 | 67 | 99 |
| On-board NIC | 84.5 | 68 | 98 |
We attempted testing in other multiplayer environments like Team Fortress 2 and World of Warcraft, but we couldn't eliminate graphics as a factor when side by side testing with different players like we could in EVE. If we did sequential testing, one run to the next had very high variability even on the same hardware (due to the influence of other players).
We did run some tests in not very highly populated areas of WoW and found that framerate and ping seemed to show no difference. This might be different for highly populated areas, but again we couldn't be very deterministic in testing this.
In trying to do the similar testing with Team Fortress 2, the Killer Xeno Pro would be faster in once instance and slower in the next. There was no real consistency to our data in this case.
Bigfoot claims that there is benefit from the hardware in games like WoW, Team Fortress 2, Counter Strike: Source, and other games with high volumes of network traffic. We really do not doubt the capability of the hardware to provide some sort of difference, but our tests just are not deterministic enough to appropriately compare the hardware. But in a way this does tell us something very important: factors other than client side networking (like the performance of the network itself, other players, servers, and potentially graphics) have a much higher impact on performance.
The Killer Xeno Pro does suggest another advantage: bandwidth prioritization and throttling. The hardware is capable of Quality of Service (QoS) like prioritization on a per application basis, and every application can have upload and download bandwidth caps. This could potentially help out when multiple network heavy applications are vying for bandwidth. We decided to test this with both EVE (for framerate and download speed) and WoW (for framerate and latency).
In our EVE test, we used uTorrent to download a 650 MB file while we played EVE. Because we had to do this test sequentially rather than side by side (the bandwidth demand from on torrenting computer would negatively impact the bandwidth available to both PCs -- a point we'll come back to later), our frame rates aren't directly comparable because of all the other player activity. Please keep in mind that fluctuations in the multiplayer environment make this a non-deterministic test despite the fact that framerates are similar.
| EVE Test + Torrent | Average FPS |
| Control (no download) | 98.7 |
| Killer Xeno Pro w/ Prioritization | 98.4 |
| On-board NIC | 98.5 |
We did, however, see a very large difference in the time it took to download our torrent.
| Torrent Time + EVE Test | Download Time in Minutes |
| Control (no game) | 27 Minutes |
| Killer Xeno Pro w/ Prioritization | 69 Minutes |
| On-board NIC | 30 Minutes |
Since we can't get an assessment of ping times in EVE, we did some testing on WoW in the same unpopulated area. Normalized to the average latency we experienced while not downloading a torrent, here's the latency incurred by downloading a torrent:
| WoW Test + Torrent | Increase in average Latency |
| Killer Xeno Pro w/ Prioritization | 15ms |
| On-board NIC | 25ms |
Even with these latency differences, our framerates were very constant at about 54 FPS with 0.4% difference between the three different setups.
Again, this might have a larger impact in a more highly populated area in WoW. But the hardware does show a ping time advantage over our on-board NIC when downloading a torrent while gaming.
Facebook
Twitter
RSS
121 Comments
View All Comments
tamalero - Saturday, July 18, 2009 - link
just wondering guys, why you didnt test on a NORMAL machine and not on a high end machine?to test the offloading, you should have used games that are heavily cpu bound or that limits cpu or similars.
a dual core machine with some heavy source based engines and see if it offloads the cpu....
if you're not gonna squish what the card promises, why even test then?
kmmatney - Friday, July 10, 2009 - link
A more useful article would be whether a sound card improves performance - I'd like to see a good article on that.hoak - Friday, July 10, 2009 - link
I hope this isn't a trend, but this review and others like it on AnandTech are not of the caliber of other of reviews past, or what's offered on other sites like X-Bit Labs where rigorous objective testing standards, science, and hard facts are the arbiter of a review, not this subjective and half-assed swill that panders to the the difficulty of testing and baseless opinion.Five Frowny Faces For AnandTech...
:( :( :( :( :(
LoneWolf15 - Thursday, July 9, 2009 - link
"Adding to this, the Killer Xeno Pro hardware is capable of offloading other network centric tasks like VoIP, firewalls, and networked storage (though these features are not all fully supported with appropriate software at this point in time)."This is the ONE use I would find for this card, rather than gaming. I know people who rely on VOiP, or teleconferencing for distance-learning and therapy who could truly benefit from what this card has to offer. And of course, most of those features aren't fully supported.
Hmm. Think I'll just make sure those people have a fast enough CPU to crunch things (which really isn't all that fast, a Core 2 Duo P8400 on a laptop can handle it no problem) and a low-latency network, since I probably can't guarantee even, consistent support with something like this product. Maybe someone like Cisco can buy the company or Killer NIC can look at the business aspect of this, since I think they'd make more money there, and then be able to use it to support the gaming side of things.
jiulemoigt - Wednesday, July 8, 2009 - link
I started reading through the comments to see if anyone posted this but the whole test was invalid. Two computers with the exact same hardware going to a location on internet could have totally different latencies because they took different routes to get to the host server.Though the results reflewct reality some times it is going to be fast some times it is going to be slow, but mostly it is dependant on how congested the net in general and how busy your server is. The network stack is not significant to transfering data unless something is broken.
Last most built in NIC have traces directly to the north or south bridge depending on the chip and don't share bandwidth with anything else.
swaaye - Friday, July 10, 2009 - link
Onboard NICs are either on PCI or PCIe and definitely do share with other such devices.Teloy - Wednesday, July 8, 2009 - link
«the Killer Xeno Pro can disable it's "game mode" and enable "application mode»; «(...)the major feature of the Killer Xeno Pro is it's ability to offload TCP/IP work(...)... "It's" is a typo only allowed to first grade children... You should pay more attention to grammar when writting an article on a website as prestigious as Anandtech...jaggerwild - Tuesday, July 7, 2009 - link
I have to question why this article was done, unless EVGA is buying space here.........Rasterman - Monday, July 6, 2009 - link
One question that would put things into perspective is how much CPU is used to do TCP/IP functions? My guess is a tiny fraction of 1%, especially with a monster CPU like in the test. Therefore even if you offload 100% of network computation, you are never going to notice a difference.It seems this card might be more useful for a web or file server. Testing it in a system with as high of a network load as possible would seem to be a much better test if the card is actually doing anything. Granted people are going to buy this for games, but the problem testing with games is they are designed to use as little network as possible.
hooflung - Monday, July 6, 2009 - link
Q: Who else would benefit from using this product?A: Among other things, Killer is a UDP (User Datagram Protocol) accelerator, so other applications that are UDP intensive could also see benefits. In addition, programmers, Linux developers, and other enthusiasts will love FNA because they will be able to write or download Flexible Network Applications that run on Killer's NPU. Flexible Network Applications could be VOIP apps, peer to peer file sharing utilities...the list is endless.
Q: Some of the articles about your card say you offload TCP/IP. Is this true?
A: While our card does perform some TCP/IP acceleration functions, it does not fully offload this network protocol (nor does any other network card in the consumer market today). Most MMO and First Person Shooter online games don't use TCP/IP as the primary protocol, but rather use UDP. Among other things, Killer IS a UDP (User Datagram Protocol) accelerator, which makes it EXTREMELY FOCUSED on gaming, and very different from what any other company is offering today.