Navigating the Impending Memory Shortage: A Look at AI's Exponential Growth

Dell CEO's Startling Forecast on AI Memory Needs

According to recent reports, Dell's Chief Executive Officer, Michael Dell, shared a concerning projection at a Bank of America event. He anticipates an astounding 625-fold increase in the overall memory requirements for the AI sector by the year 2028, compared to 2022 levels. This dramatic surge is attributed to a dual factor: the enhanced memory capacity of individual AI accelerators and the simultaneous expansion in the scale of systems within AI infrastructure.

The Escalating Demands of AI Accelerators

The foundation of this forecast lies in the evolution of AI accelerators. In 2022, a prominent AI accelerator like Nvidia's H100 was equipped with 80 GB of High Bandwidth Memory (HBM3). However, by 2028, it's projected that accelerators will integrate up to 2 TB of HBM, representing a more than 25-fold increase in DRAM per unit. This exponential growth in individual component memory is a primary driver of the overall market demand.

The Proliferation of AI in Data Centers

Compounding the individual accelerator's memory expansion is the projected increase in the deployment rate of these AI accelerators within data centers. Dell estimates that the number of AI accelerators in data centers will multiply by a factor of 25 over the same period. When these two growth factors—individual memory capacity and deployment scale—are combined, they lead to the staggering 625-times increase in total memory demand.

Revisiting the Calculations: A Closer Look at Specific Implementations

While the 625x figure is a striking headline, a more granular analysis, considering the practical implementation of advanced AI chips like Nvidia's Vera Rubin, suggests a slightly different, though still substantial, growth. For instance, a single rack might incorporate 576 GB of HBM4, which is approximately 7.2 times more memory than an H100 accelerator. If the deployment rate remains consistent with Dell's predictions, the total memory demand would still climb by a factor of 180, indicating a massive, albeit somewhat less extreme, increase.

The Oligopoly of HBM Production and Supply Chain Challenges

The current global manufacturing landscape for HBM4 is highly concentrated, with only three major players: SK hynix, Samsung, and Micron. Even as other companies strive to catch up with HBM3 offerings, the existing producers are already struggling to meet current memory demands. The projected surge in demand over the next few years is expected to place immense pressure on this limited supply chain, potentially exacerbating existing shortages.

Broader Implications Beyond HBM: LPDDR5x and NAND Flash

The impending memory shortage is not confined solely to HBM. The ripple effect is expected to impact other critical memory types, including LPDDR5x, commonly found in laptops and handheld devices, and NAND flash storage. Modern AI servers, such as the Nvidia GB200 NVL72, demand significant quantities of LPDDR5x (e.g., 480 GB per compute tray) and massive amounts of flash storage (hundreds of terabytes per rack), further straining the supply of these components.

The Future Outlook: Balancing Demand and Supply

The industry faces a formidable challenge in scaling up memory production to match AI's voracious appetite. While leading memory manufacturers are planning new production facilities, it remains uncertain whether these expansions will be sufficient to prevent critical shortages. The hope is that manufacturing growth can at least maintain the current "affordable but expensive" status quo for high-end memory. However, should the demand-supply imbalance worsen significantly, the consequences for various technology sectors, including PC gaming, could be severe, impacting hardware accessibility and cost.