EVGA Killer Xeno Pro: The Impact of Network Offloading
by Derek Wilson on July 3, 2009 4:20 AM EST- Posted in
- Networking
Mostly Deterministic Testing
Designing tests to determine the real world benefit of the Killer Xeno Pro has proven quite difficult. Even though frame rate testing with single player games isn't strictly deterministic, proper tests can produce results that are fairly consistent and have low variance. We haven't included many MMOs or multiplayer games that don't utilize timedemo functionality in our graphics hardware tests specifically because they are very hard to appropriately benchmark. We can get ideas about performance from play testing, but graphs and charts have a certain finality and authority to them that we just don't want to lend to tests that we can't stand behind are representative of relative performance.
We did come up with one test that is highly reliable, however. This test is a side by side comparison of framerate when playing EVE online. We ran two different computers side by side with exactly the same hardware and software setup except that we installed the Killer Xeno Pro in one box. Both instances of EVE undocked characters in Jita (a system that typically hosts about 1000 players at a time) and flew to nearly the same spot. Because EVE allows players to choose something to "look at" and centers the camera on that object, were were able to have two instances of the game running with players very near each other (requiring very similar network data) and with exactly the same graphical load (because they were looking at the same thing).
Our EVE test is in a place where there were a very high number of other players and we were able to eliminate as many other factors as possible from testing. This test showed no difference in performance with or without the Killer Xeno Pro:
| EVE Test | Average FPS | Min FPS | Max FPS |
| Killer Xeno Pro | 84.3 | 67 | 99 |
| On-board NIC | 84.5 | 68 | 98 |
We attempted testing in other multiplayer environments like Team Fortress 2 and World of Warcraft, but we couldn't eliminate graphics as a factor when side by side testing with different players like we could in EVE. If we did sequential testing, one run to the next had very high variability even on the same hardware (due to the influence of other players).
We did run some tests in not very highly populated areas of WoW and found that framerate and ping seemed to show no difference. This might be different for highly populated areas, but again we couldn't be very deterministic in testing this.
In trying to do the similar testing with Team Fortress 2, the Killer Xeno Pro would be faster in once instance and slower in the next. There was no real consistency to our data in this case.
Bigfoot claims that there is benefit from the hardware in games like WoW, Team Fortress 2, Counter Strike: Source, and other games with high volumes of network traffic. We really do not doubt the capability of the hardware to provide some sort of difference, but our tests just are not deterministic enough to appropriately compare the hardware. But in a way this does tell us something very important: factors other than client side networking (like the performance of the network itself, other players, servers, and potentially graphics) have a much higher impact on performance.
The Killer Xeno Pro does suggest another advantage: bandwidth prioritization and throttling. The hardware is capable of Quality of Service (QoS) like prioritization on a per application basis, and every application can have upload and download bandwidth caps. This could potentially help out when multiple network heavy applications are vying for bandwidth. We decided to test this with both EVE (for framerate and download speed) and WoW (for framerate and latency).
In our EVE test, we used uTorrent to download a 650 MB file while we played EVE. Because we had to do this test sequentially rather than side by side (the bandwidth demand from on torrenting computer would negatively impact the bandwidth available to both PCs -- a point we'll come back to later), our frame rates aren't directly comparable because of all the other player activity. Please keep in mind that fluctuations in the multiplayer environment make this a non-deterministic test despite the fact that framerates are similar.
| EVE Test + Torrent | Average FPS |
| Control (no download) | 98.7 |
| Killer Xeno Pro w/ Prioritization | 98.4 |
| On-board NIC | 98.5 |
We did, however, see a very large difference in the time it took to download our torrent.
| Torrent Time + EVE Test | Download Time in Minutes |
| Control (no game) | 27 Minutes |
| Killer Xeno Pro w/ Prioritization | 69 Minutes |
| On-board NIC | 30 Minutes |
Since we can't get an assessment of ping times in EVE, we did some testing on WoW in the same unpopulated area. Normalized to the average latency we experienced while not downloading a torrent, here's the latency incurred by downloading a torrent:
| WoW Test + Torrent | Increase in average Latency |
| Killer Xeno Pro w/ Prioritization | 15ms |
| On-board NIC | 25ms |
Even with these latency differences, our framerates were very constant at about 54 FPS with 0.4% difference between the three different setups.
Again, this might have a larger impact in a more highly populated area in WoW. But the hardware does show a ping time advantage over our on-board NIC when downloading a torrent while gaming.
Facebook
Twitter
RSS
121 Comments
View All Comments
james jwb - Friday, July 3, 2009 - link
Integrated NIC's are the de-facto standard in use by Bigfoot's target audience. Almost no one even thinks of looking for an alternative to what comes on there motherboard. We have always assumed the intergrated, free NIC's are more than adequate. This is not the sound card market where in the high-end, people automatically look at add-in boards like the X-fi or Xonar. If Bigfoot wants to sell a number of these cards it's going to have to offer something over the free, integrated NIC's, not add-in cards.This review tells me that for what the average home user does with their PC, it offers pretty much nothing, as most suspected. To me this stinks of an idea that sounded good on paper, in practice failed to deliver any adequate performance increase for the average user, and has only been funded because sometimes bad ideas marketed well can still create profits. Unless Bigfoot can offer something tangible here, I'd honestly like to see them go the way of the dinosaurs (Yes, a big meteor lands on their HQ).
But hey, maybe in the future they can always try the Fatal1ty brand and really go to town :)
Maybe I'm being a bit harsh, but i want something more for my money than marketing and geeky technical spec's for handling something no better (in performance terms) than the boring software TCP/IP stack
Qi - Friday, July 3, 2009 - link
But if no comparison between the Killer NIC and other add-in NICs is made, we don't know which add-in NIC is the best. Also, other add-in NICs are substantially cheaper. Take these for example:Intel Gigabit CT Desktop Adapter
Intel PRO/1000 PT Desktop Adapter
Intel PRO/1000 GT Desktop Adapter
You can get these cards for around $30.
Anonymous Freak - Saturday, July 4, 2009 - link
That's because those cards have the same chip as the onboard Gigabit Ethernet on many Intel-based boards. (Or at least, the same family of chip, with the same features. Deep down, it's probably the same silicon; just named different for the different interfaces.)lyeoh - Sunday, July 5, 2009 - link
Just claiming the cards have the same chip doesn't mean a thing.Lets see better benchmarks against integrated NICs and other add-on NICs. CPU usage, packets per second, throughput, max latency under various conditions.
This review as it is isn't very useful.
In my opinion if you want a better gaming experience you might as well use the extra money to get a fancy router and use it to squish torrent speeds down (and maybe even force torrents to use smaller packets, if you really want lower latency at the cost of throughput).
A smarter router is even more useful if you are sharing the connection with other computers, since it can also help control traffic to those other computers. Whereas this expensive NIC won't help.
mindless1 - Saturday, July 4, 2009 - link
Do you know this factually or are you just guessing? Most boards do not use the same chip, and "family" means little, just a generational grouping. By saying they have the same features you are only saying they all have network features and of course they would.Deep down, saying it's the same silicon would be like saying all silicon is the same, all NICs were identical which they are not.
yyrkoon - Saturday, July 4, 2009 - link
Yeah well, "Those cards" are also the best Ethernet cards out there. They offer ToE, Link Agregrigation, and all the other goodies a network guru would want. I own an Intel Pro 1000 PT, and I can say with confidence that there is no way I am going to spend 6x as much money on a NiC, that will only perform *maybe* as good as the $20 NiC I got on sale.Souka - Saturday, July 4, 2009 - link
Here's a question...I have a c2d mid-level system..using on-board Marvell Yukon 88E8xxx gigabit adapter.
I have an old Intel Pro/1000 GT PCI (not PCI-e) adapter sitting in a box.... would it benifit me to toss it in?
Same question goes for my sound... Should I use, onboard ADI AD1988B 8-channel High Definition Audio CODEC or my old Creative PCI Fata1ity Pro card?
Always apprecaite constructive advice..ideas... :)
Thanks!
yyrkoon - Saturday, July 4, 2009 - link
Maybe, but probably not. The Marvell Ethernet is possibly tied into the PCI-E bus of the motherboard, which means that it *may* not be sharing the bus bandwidth with other peripherals, CPU, and memory; or it could be. The PCI bus is flawed in that way; e.g. it shares bandwidth with all slots, PLUS memory, and CPU. The PCI-E bus in theory is not, but it does not always work that way.The best way to find it is try them both. The Yukon without the PCI card in, and the PCI card with the integrated Ethernet disabled in the BIOS. Myself, I test with the application I have in mind , but you'll probably have many naysayers say something along the lines of "No! you must use this test app" or whatever. Test apps are great if that is all you're going to do ( test ), but if all you want is XX amount of MB/s coming from your XFS + Samba box, well that is all that matters right ?
One thing that I have found out over the course of *many* tests is that file block size can play a big factor in a lot of cases when testing for file transfers with many protocols. A lot of those times, if you pay attention you will see a pattern emerge. The problem here is that disk block sizes are often much larger than your Ethernet card can handle without breaking down into smaller chunks. Anyhow, to keep this reasonably short, unless you're directly transferring from a large RAM disk, you're going to see a performance hit no matter what between disks and Ethernet. That is where ToE is supposed to come into play, but it does not always work as some would have you believe. The idea is that ToE offloads that processing from the CPU keeping your CPU from bogging down. Still there is lots of possible processing going on, so you're likely to still see a performance hit, albeit probably less.
Hope I did not bore you too much . . .
Souka - Sunday, July 5, 2009 - link
not at all...thanks. What you said makes sense.I might play with it when I have some time.
Sound card will be easy to test via game benchmarks/timedemos...but network card I'll probably just use file copy tests if different type/sizes to see how the NIC (onboard vs PCI) makes a difference.
thx again
yyrkoon - Tuesday, July 7, 2009 - link
I would like to add that any add in sound card should perform better than integrated sound. I did a bunch of testing with Oblivion, and even my old 16 bit Sound blaster !live card made about 5 FPS difference. Now, keep in mind that if you have AGP graphics, that this would share the bandwidth from the PCI bus as well . . . So, if all you have is a single sound card using the PCI bus, you *should* be golden. Also, know that PATA HDD's can use the PCI bus for data as well. But typically, if all your peripherals use less than 133MB/s theoretical you should be ok. Theoretical meaning even PCI communications have overhead too, just like Ethernet and the SATA protocols.