AMD announced that its MI325X GPU would be released later this year and that it beats NVIDIA’s H200 GPUs on several fronts.
NVIDIA is the current leader in data center chip sales, estimated to hold over 70% of the market share for AI chips, but Intel and AMD are making strides with their own offerings.
At Taiwan’s Computex 2024 event, AMD CEO Lisa Su announced that the Instinct MI325X GPU would be released later this year. AMD says that its successor to the MI300 will feature more memory and faster data throughput.
AMD is gaining wider adoption for its data center chips, with companies like Microsoft incorporating them into their Azure cloud services while Microsoft, Meta, and Oracle have all adopted the MI300 platform. The company is targeting $4b in sales for 2024.
Su said the MI325X significantly outperforms NVIDIA’s popular H200 GPU in memory capacity, bandwidth, and performance.
The MI325X features up to 288 GB of HBM3e memory and 6 TBps of bandwidth. That’s more than twice the memory NVIDIA’s H200 has with 30% faster data throughput.
It achieves 2.6 petaflops peak theoretical throughput for 8-bit floating point (FP8), and 1.3 petaflops with 16-bit floating point (FP16). That’s 30% higher than the H200.
A single server made up of eight of these GPUs combined on the Instinct MI325X Platform will have enough memory to run advanced AI models of up to 1 trillion parameters, double the size supported by an H200 server.
Today at Computex we unveiled a multiyear, expanded AMD Instinct accelerator roadmap which will bring an annual cadence of leadership AI performance and memory capabilities at every generation. pic.twitter.com/PQKyVAnAVW
— AMD (@AMD) June 3, 2024
While NVIDIA’s H200 is its flagship GPU that is currently available, Su didn’t mention how the MI325X would stack up against the Blackwell GPUs NVIDIA will start shipping later this year.
Top-end Blackwell models will have up to 192GB of HBM3e memory with 8 TB/s bandwidth. That’s a fair amount less memory but more bandwidth than AMD’s top offering.
AMD says that it will ramp up new GPU development with a new family of GPUs expected to be released every year. That’s the same cadence that NVIDIA CEO Jensen Huang said NVIDIA is aiming for.
In 2050 AMD will release the MI350 which will use a 3nm process and will use AMD’s CDNA 4 architecture.
Su says the CDNA 4 architecture will be a generational leap in AI computing which will deliver a 35x increase in inference performance over its current CDNA 3 architecture.
Shifting all that data between GPU clusters and servers needs high-speed networking. Su concluded her address by saying that “the future of AI networking must be open.”
She announced that last week, AMD joined a consortium of high computing companies with the aim of developing a high-bandwidth, low-latency networking standard to connect hundreds of AI accelerators.
NVIDIA wasn’t invited to participate in the project and has its own proprietary standard. Su said that the UALink standard will be a “great alternative to proprietary options.”
The AI GPU battle lines have been drawn with very little subtlety and NVIDIA must be starting to feel the heat from AMD and Intel.
