AMD has launched the Instinct MI325X AI accelerator, specifically designed to provide high-performance support for AI inference. Built on advanced technology, the accelerator meets the demands of generative AI models in edge computing scenarios.
Core Specifications
The Instinct MI325X is based on the AMD CDNA 3 architecture and comes with 256GB of HBM3E memory, delivering a memory bandwidth of up to 6.0TB/s. Compared to the NVIDIA H200, it provides 1.8x more memory capacity, 1.3x more bandwidth, and a 1.3x improvement in peak theoretical FP16 and FP8 compute performance. These specifications enable it to efficiently process large-scale AI models.
Edge Computing Optimization Features
This accelerator is specially optimized for edge computing scenarios, enabling high-efficiency execution of AI inference tasks in resource-constrained environments. The high-capacity memory design supports low-latency processing, facilitating the deployment of complex generative AI applications at edge nodes.
Benchmark Performance
In inference benchmarks using models like Mistral-7B, the MI325X has demonstrated higher throughput. Official data from AMD confirms its leading performance in generative AI tasks, making it suitable for both data centers and edge infrastructure.
Release and Availability Plan
AMD plans to begin production shipments of the MI325X in Q4 2024. Starting in Q1 2025, systems featuring the accelerator will be launched by platform providers such as Dell Technologies, Hewlett Packard Enterprise, Lenovo, and Supermicro, ensuring broad availability.
