OpenAI and NVIDIA's Alliance Strains as AI Shifts Focus to Inference

OpenAI and NVIDIA (NVDA-US) are navigating a strained strategic relationship as artificial intelligence shifts from model training to inference, where speed and efficiency are critical. While NVIDIA dominates AI training with its GPUs, its reliance on external memory creates bottlenecks in high-speed inference tasks—particularly for OpenAI’s latency-sensitive Codex coding product. OpenAI is now evaluating alternatives, including chips from Cerebras and AMD, aiming to internalize roughly 10% of its inference workload. This pivot has stalled NVIDIA’s planned investment in OpenAI, with negotiations deadlocked for months due to shifting hardware requirements. NVIDIA countered by securing an exclusive licensing deal with inference startup Groq, blocking a potential OpenAI partnership. Despite public reassurances from CEOs Jensen Huang and Sam Altman affirming mutual support and ongoing collaboration, the underlying tension reflects a deeper realignment in the AI infrastructure race.