The competition to accelerate AI inference is heating up, and Cerebras received a warm reception from markets when it went public in May with its specialized chips. However, French startup Kog is betting that conventional GPUs still hold significant untapped potential. In May, the company gained attention on Hacker News' front page with a technical preview demonstrating that "extremely fast single-request decoding is possible on the standard datacenter GPUs enterprises already own," using AMD MI300X and NVIDIA H200 GPUs for the demo. While some were let down that this didn't apply to consumer laptops, others recognized the opportunity. As inference speed and cost have become critical constraints, Kog's approach—unlocking new capabilities on existing hardware through software optimization—drew more than just curiosity.

Based on early interest, the solo founder believes software engineering will be the first practical use case. Experienced users of Claude Code know the frustration of waiting hours for results. Anthropic itself acknowledges that speed has value, charging a premium for Claude's Fast Mode. Kog aims to attract customers deterred by these delays, particularly those relying on AI workflows for professional tasks. The startup also has design partners who generate games and apps from prompts, and for them, faster output via the Kog Inference Engine (KIE) would directly boost revenue, according to Delalleau. Still, the company recognizes the market isn't fully mature yet. Through observing demand, Kog discovered that potential customers aren't willing to fine-tune small models. "And that's why since the launch, we've been fully focused on accelerating the development of larger models to meet the demand we've seen."

This ambition means Kog has a significant hurdle to clear in delivering on its promise of "30x faster LLM inference." Its demo showcased an impressive 3,000 tokens per second (TPS) per request—but it used a purpose-built small model with only around 2 billion parameters, the now open-sourced Laneformer 2B. Despite skeptics, Delalleau is confident the same methodology can work effectively with large language models, whose size often poses challenges for inference hardware. "GPUs have a bright future," he said. For Kog's CEO, the belief that GPUs aren't suited for decoding has become a misconception; newer GPUs come with increasing memory bandwidth just waiting to be utilized.

Kog isn't the only one betting that software optimization can push GPUs beyond their advertised capabilities. ZML, another French company, released hardware-agnostic software that bypasses Nvidia's CUDA to enable fast inference across competing chips. But Delalleau says Kog is more comparable to Stanford's Hazy Research lab, with an even deeper focus on GPU acceleration. Delalleau himself isn't a researcher, and his first startup—TechCrunch50 2009 alum Stribe—has little in common with his current venture, aside from his former cofounder turned VC Kamel Zeroual, whose firm Varsity VC co-led Kog's seed round. The startup's deep technical focus stems from Delalleau's unusual background. After studying solid-state physics at France's École Polytechnique, he moved into offensive cybersecurity, also known as white hat hacking. According to Delalleau, this shaped the mindset he now encourages in his team. On the science side, "there's this mindset of understanding the laws of physics, and the laws of the GPU in order to make the most of them."

As for hacking, the four-time finalist at DEF CON's CTF tournament says it taught him "to reverse-engineer things at a very low level—down to assembly language and binary code—to understand how it works, and to try to use it to achieve a goal for which it wasn't necessarily designed."

The downside is that this approach is highly hands-on and time-intensive. "For every new GPU, we'll dedicate several weeks or even months, to really dig into the details and conduct GPU engineering research on that hardware." With a team of just 11 people, this limits how many chips Kog can support, at least in the near term. Looking ahead, Kog hopes to integrate its methodology into agent-based pipelines, allowing it to cover more chips and models. As Europe works to build its own capabilities in these areas, this could provide sovereign-tech tailwinds for the startup, which already has support from Scaleway and backing from France's Bpifrance and the French Tech 2030 program. For now, though, Kog must prove its approach works on LLMs—a crucial step for attracting more funding. "Once we've implemented our first major model at 10x speed, which I think will be in September, we'll be able to start demonstrating customer traction and from there, raise our Series A," Delalleau said.