Groq Weighs 3D-Stacked DRAM for Next-Gen AI Chips in First Move Under Nvidia

Groq is evaluating 3D-stacked DRAM packaging for its next-generation AI chips, its first reported architectural move since being effectively acquired by Nvidia, according to foreign media reports. The shift would move the company away from its SRAM-dependent design toward higher memory capacity. The proposed "3DS" approach uses through-silicon via (TSV) technology to stack multiple dies vertically, drilling thousands of micro-vias through DRAM chips to deliver high memory density per unit area. Unlike HBM, which prioritizes extreme data exchange rates, 3DS keeps existing DRAM standard interface protocols while focusing on capacity expansion. Groq's LPU architecture has drawn attention for high inference speed, but its small SRAM capacity limits performance on models with hundreds of billions of parameters. DRAM's native read-write speed is slower than SRAM, but its larger capacity is an advantage for complex model deployment. The 3D stacking plan aims to preserve speed while resolving capacity constraints for very large AI inference workloads.