EVGA Killer Xeno Pro: The Impact of Network Offloading
by Derek Wilson on July 3, 2009 4:20 AM EST- Posted in
- Networking
Mostly Deterministic Testing
Designing tests to determine the real world benefit of the Killer Xeno Pro has proven quite difficult. Even though frame rate testing with single player games isn't strictly deterministic, proper tests can produce results that are fairly consistent and have low variance. We haven't included many MMOs or multiplayer games that don't utilize timedemo functionality in our graphics hardware tests specifically because they are very hard to appropriately benchmark. We can get ideas about performance from play testing, but graphs and charts have a certain finality and authority to them that we just don't want to lend to tests that we can't stand behind are representative of relative performance.
We did come up with one test that is highly reliable, however. This test is a side by side comparison of framerate when playing EVE online. We ran two different computers side by side with exactly the same hardware and software setup except that we installed the Killer Xeno Pro in one box. Both instances of EVE undocked characters in Jita (a system that typically hosts about 1000 players at a time) and flew to nearly the same spot. Because EVE allows players to choose something to "look at" and centers the camera on that object, were were able to have two instances of the game running with players very near each other (requiring very similar network data) and with exactly the same graphical load (because they were looking at the same thing).
Our EVE test is in a place where there were a very high number of other players and we were able to eliminate as many other factors as possible from testing. This test showed no difference in performance with or without the Killer Xeno Pro:
| EVE Test | Average FPS | Min FPS | Max FPS |
| Killer Xeno Pro | 84.3 | 67 | 99 |
| On-board NIC | 84.5 | 68 | 98 |
We attempted testing in other multiplayer environments like Team Fortress 2 and World of Warcraft, but we couldn't eliminate graphics as a factor when side by side testing with different players like we could in EVE. If we did sequential testing, one run to the next had very high variability even on the same hardware (due to the influence of other players).
We did run some tests in not very highly populated areas of WoW and found that framerate and ping seemed to show no difference. This might be different for highly populated areas, but again we couldn't be very deterministic in testing this.
In trying to do the similar testing with Team Fortress 2, the Killer Xeno Pro would be faster in once instance and slower in the next. There was no real consistency to our data in this case.
Bigfoot claims that there is benefit from the hardware in games like WoW, Team Fortress 2, Counter Strike: Source, and other games with high volumes of network traffic. We really do not doubt the capability of the hardware to provide some sort of difference, but our tests just are not deterministic enough to appropriately compare the hardware. But in a way this does tell us something very important: factors other than client side networking (like the performance of the network itself, other players, servers, and potentially graphics) have a much higher impact on performance.
The Killer Xeno Pro does suggest another advantage: bandwidth prioritization and throttling. The hardware is capable of Quality of Service (QoS) like prioritization on a per application basis, and every application can have upload and download bandwidth caps. This could potentially help out when multiple network heavy applications are vying for bandwidth. We decided to test this with both EVE (for framerate and download speed) and WoW (for framerate and latency).
In our EVE test, we used uTorrent to download a 650 MB file while we played EVE. Because we had to do this test sequentially rather than side by side (the bandwidth demand from on torrenting computer would negatively impact the bandwidth available to both PCs -- a point we'll come back to later), our frame rates aren't directly comparable because of all the other player activity. Please keep in mind that fluctuations in the multiplayer environment make this a non-deterministic test despite the fact that framerates are similar.
| EVE Test + Torrent | Average FPS |
| Control (no download) | 98.7 |
| Killer Xeno Pro w/ Prioritization | 98.4 |
| On-board NIC | 98.5 |
We did, however, see a very large difference in the time it took to download our torrent.
| Torrent Time + EVE Test | Download Time in Minutes |
| Control (no game) | 27 Minutes |
| Killer Xeno Pro w/ Prioritization | 69 Minutes |
| On-board NIC | 30 Minutes |
Since we can't get an assessment of ping times in EVE, we did some testing on WoW in the same unpopulated area. Normalized to the average latency we experienced while not downloading a torrent, here's the latency incurred by downloading a torrent:
| WoW Test + Torrent | Increase in average Latency |
| Killer Xeno Pro w/ Prioritization | 15ms |
| On-board NIC | 25ms |
Even with these latency differences, our framerates were very constant at about 54 FPS with 0.4% difference between the three different setups.
Again, this might have a larger impact in a more highly populated area in WoW. But the hardware does show a ping time advantage over our on-board NIC when downloading a torrent while gaming.
Facebook
Twitter
RSS
121 Comments
View All Comments
DerekWilson - Friday, July 3, 2009 - link
So ... I looked it up ... (search for something like wow port usage or wow port forwarding or something)It seems that everywhere I look, the internet tells me that WoW uses TCP over port 3724 ...
I looked up EVE and it was a little harder to find info on -- but it looks to me like it uses both TCP and UDP for different things. Here's what I saw:
UDP ports 26001, 3478 and 5060-5062
TCP ports 26000, 80 and 443
EVE definitely uses UDP for it's voice support and it seems like it requires both UDP and TCP ports for other game data.
...
Additionally, the Killer Xeno Pro software only detects applications that use UDP and not applications that use only TCP ... so it makes sense that if it cannot detect WoW that WoW would be using TCP ... and if it detects EVE then EVE must be using UDP for at least something.
If you still think WoW uses UDP and EVE uses only TCP then please post links to your sources ...
mesiah - Saturday, July 4, 2009 - link
Strange how the wise ass know it alls disappear right after you hit them with facts. Its one thing for someone to come in and inform you that you got a point wrong in your review and back it up with sources. Its another for you trolls to show up, tout your epic knowledge you got from "the dude that made this shit." and then spit on the people that took the time to do the review. First, what is the point of reading the article if the only reason you are here is to give the writer grief? And second, If you thought you could do a better job maybe you should write you own articles so we can come and piss in your cheerios.Flawed or not, thanks for taking the time to do the article Derek and show us some real world numbers.
crimson117 - Friday, July 3, 2009 - link
I looked it up too!WoW similarly uses TCP for gameplay and UDP for voice support:
From http://us.blizzard.com/support/article.xml?locale=...">http://us.blizzard.com/support/article....cleId=21...
What do I need to know about ports?
Anytime your computer receives incoming data, it is sent to a "port". Your computer has many ports that can receive data, and different activities will utilize different ports. World of Warcraft & Burning Crusade use TCP port numbers 1119 and 3724 to play, and UDP port 3724 for in game Voice chat. The Blizzard Downloader, which downloads patches, also uses TCP ports 6112 and the range 6881-6999. For walkthroughs on router and firewall configuration you can use the Networking Help for the Blizzard Downloader page.
ShannonG - Saturday, January 30, 2010 - link
It is hard to believe any major MMORPG uses TCP for situational updates. Logging in, updates, billing, web, etc... sure.But for for game updates? 90% of it is real-time and redundant.
I don't play WoW, but if you routinely experience "warping" now you know why - craptastic network architecture.
A MMORPG with a well-designed network infrastructure will use a [custom] selectively-reliable UDP protocol, colloquially referred to as "RUDP".
If the card actually could/does off-load the networking stack [including firewalling et. al.] you stand to recapture 5%-10% of the CPU if it is bandwidth intensive.
Most games are not bandwidth intensive, quite the opposite; and it cannot significantly improve latency - that latency delays of the Internet will swamp the latency delays of packet delivery (ms vs us).
What this card will do is move the packet processing from whatever system bus your NIC is currently on to the ePCI bus. That's probably not a good thing either - the video card is on that bus.
Stas - Friday, July 3, 2009 - link
Given the return on the investment, I would pay $25 for this NIC at the most. Not $100+ (shit, I might as well go for an Intel dual Gigabit LAN NIC, if I'm to spend over $100).DerekWilson - Friday, July 3, 2009 - link
There is typically a baseline cost to add-in network hardware ... if you need something to put in your box, you'll probably spend at least $25-$30 just to get something equivalent to what's on most motherboards.bigboxes - Friday, July 3, 2009 - link
I just replaced my gigabit card on my file server with a new Linksys gigabit card. $30. No, my mobo only had 10/100, so I had to purchase the card. I remeber that D-Link's was $25 and Netgear was $20. The U.S. Robotics card was $15, but seeing as that was the card that just failed I tried the Linksys route.DerekWilson - Friday, July 3, 2009 - link
So maybe $20 - $30 ... :-) but still, you've got to pay something for just the PCB, the port, and the chips ... I certainly agree that for what it delivers in realized performance the $100 premium is too much for the Killer Xeno Pro ... but it is definitely more reasonable than their first offering.Shadowmage - Friday, July 3, 2009 - link
This is a horribly misleading article. The claim is that the card is better than standard networking cards, yet the author never tests the card against its competitors - add-in card NICs.Would you test a new graphics card against integrated graphics?
CptTripps - Tuesday, July 7, 2009 - link
How is it misleading? The author states that even tested against an integrated NIC there was no noticable difference. He then suggested we save our money.The "Claim" comes from the manufacturer and the result posted in the article is what I expected.