On Wednesday, OpenAI introduced its first custom-built inference processor, developed in collaboration with Broadcom. Dubbed Jalapeño, this new chip is tailored specifically for the demands of OpenAI’s inference systems. The company noted that its own AI models contributed to the chip's design. Although the processor is still undergoing testing, early results indicate a notable improvement in performance-per-watt compared to existing state-of-the-art options. The partnership was formally revealed in October, though rumors about OpenAI’s chip initiatives had circulated for some time as a strategy to lessen reliance on Nvidia’s GPUs. Tech giants like Google and Amazon have also created custom chips, often referred to as “AI accelerators,” which are specialized silicon designed to expedite machine learning tasks. OpenAI president Greg Brockman detailed the company’s chip development strategy on its internal podcast shortly after the Broadcom collaboration was announced. “We have a deep understanding of the workload,” Brockman shared during the episode. “We’ve really been looking for specific workloads that are underserved, [and asking] how can we build something that will be able to accelerate what’s possible.”

Jalapeño is engineered specifically for inference, the process of executing pre-trained AI models in response to user inputs. In the announcement, OpenAI highlighted the chip’s low operational costs when handling real-time coding models. It’s likely that more resource-intensive tasks, such as pre-training, will continue to depend on Nvidia hardware. However, even modest reductions in inference expenses could significantly boost the company’s profitability. Optimizing the inference system may become a critical factor in the economics of AI moving forward, and this optimization is expected to occur across every layer of the technology stack. OpenAI is already developing agentic products like Codex and the models that underpin them, along with data centers to host these models. By moving into purpose-built chips, the company can extend this integration even further, as outlined in its announcement. “OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience,” the company wrote. “Because OpenAI operates across the stack, each layer can be optimized around the same goal: making its models faster, more reliable, and more affordable for users.”