The NVIDIA Titan V Preview - Titanomachy: War of the Titans

First Thoughts

Given what we’ve seen over the past half-decade or so with NVIDIA’s success in the GPU computing space, it’s easy to see why NVIDIA has made the architectural decisions that have shaped the Pascal and now Volta architectures. At the same time however, the increasing specialization of NVIDIA’s compute-centric GPUs and resulting products has meant that the company has needed to rethink and refocus the Titan family of video cards. Though originally created for a mixed prosumer market, Titan X and later have seen increasing success not as graphics cards but as workstation-class compute cards as professional customers have invested heavily in deep learning technologies. As a result the Titan line itself has been drifting more and more towards being a compute product, and with the launch of the Titan V, NVIDIA’s latest card takes its biggest step yet in this regard.

By switching the Titan family over from NVIDIA’s consumer GPUs to their compute-centric GV100 GPU, NVIDIA has centered the Titan V on compute more strongly than any Titan before it. And in compute it flies. Though we’ve still only had a chance to scratch the surface of Titan V, GV100, and the Volta architecture in today’s preview, all of our early benchmark data points to something that datacenter customers have already known for several months now: GV100’s not just a bigger NVIDIA GPU, and Volta’s not a minor iteration on Pascal. Sure, GV100 is also literally huge, but under the hood Volta brings a great deal of new architectural improvements and optimizations that, until the launch of the Titan V (and the opportunity to get our hands on the card) we haven’t really had the chance to properly appreciate.

The biggest factor here for NVIDIA is of course the tensor cores. NVIDIA has bet big on what’s essentially a highly specialized dense matrix multiplier, and because it maps so well to the needs of neural network training and execution, this is a bet that appears to already be paying off in spades for the company. As we saw in our GEMM results, if you can really put the tensor cores to good use, then the sheer mathematical throughput is nothing short of immense. But – and this is key – developers need to be able write their software around those tensor cores to get the most out of the Titan V. And I’m unsure whether any of this will translate into the consumer space and use cases, or if this is a feature that’s ultimately going to be mostly adopted by the HPC crown.

However even disregarding the tensor cores, what we’re seeing with the Titan V is that in terms of compute efficiency, this card appears to be on a very different level from the Pascal-based Titan Xp, and that card was no slouch either. I feel like there’s a bit of an apples-to-oranges comparison here since under the hood the cards are so different – the consumer GP102 GPU versus the compute GV100 GPU – but they are still very much Titans. Titan V is punching above its weight in many compute tasks compared to what it should be able to do on paper, and this is thanks in large part to the efficiency gains we’re seeing. For our full review hopefully we can track down a Quadro GP100 and try to better isolate how much of Volta and GV100’s gains come from improvements versus the Pascal architecture, and how much is from the unique, compute-centric configurations of NVIDIA’s Gx100-class GPUs.

For compute customers then, the flip side to all of this is that at $3000, the Titan V is quite the expensive card. Much more so than the mere $1200 of the Titan Xp. As the Titan V doesn’t experience dramatic performance gains in every application, I suspect that Titan compute customers are going to continue picking up both cards for some time to come. But a crazy as a $3000 price tag is, for customers that need to maximize single-GPU performance and/or get their hands on a product with tensor cores, then the Titan V is certainly going to fill this role well. Titan V can absolutely deliver the kind of performance to justify its price in the compute space.

However for graphics customers, we’re looking at a different story. To preface this, I don’t want to entirely discount the Titan V as a video card here, because at the end of the day it’s the fastest video card released by NVIDIA to date, gaming and all. But the immense performance gains we saw in our compute benchmarks – the kind of gains that justify a $3000 price tag – are not present in our gaming benchmarks. Right now the Titan V is moderately faster than the Titan Xp in gaming workloads – and in a world where multi-GPU scaling is on the rocks, that’s a huge element in favor of the Titan V – but that’s counterbalanced by the fact that in some cases the Titan V can’t even beat the Titan Xp.

Now how much of that is because of GV100’s suitability for gaming or how much of it is due to drivers is something that remains to be seen. The Titan V is a new card, the first graphics card based on the Volta architecture, and it’s not at all clear whether NVIDIA’s launch drivers are fully optimized for the new architecture. What is obvious is that the current drivers do have some notable bugs, and that does lend credit to the idea that the launch drivers aren’t fully baked. But even if that’s the case, that’s likely not something worth betting on. The Titan V may use the GeForce driver stack, but mentally it’s better compartmentalized as a compute card that can do graphics on the side, rather than the jack-of-all-trades nature that has previously defined the Titan family.

Closing out today’s preview, it’s hard not to come away impressed with what we’ve seen. But we’ve also just scratched the surface over the span of a few days. We’re going to continue looking at the Titan V, including getting some deep learning software up and running. Which is not to say that Titan V is merely a deep learning accelerator – it’s clearly much more than that – but this is definitely one of the most practical uses for the hardware right now, especially as software developers just now get their hands on Volta. Though for that matter I think it’ll be interesting to see what academics and researchers do with the hardware; they were one of the driving forces behind the original GTX Titan, and it’s no mistake that NVIDIA has a history of giving away Titans to these groups. Part of the need for a card like the Titan V in NVIDIA’s lineup is to help seed Volta in this fashion, so the launch of the Titan V is part of a much larger and longer-term plan by NVIDIA.