AI agents are growing increasingly advanced, transitioning from simply answering questions to independently carrying out multi-step, intricate tasks. However, before these agents can be entrusted with responsibilities like booking travel or performing financial analysis for users, model developers and the startups building such agents need assurance that they will perform reliably across a wide array of scenarios. AI labs often rely on benchmarks to demonstrate their models' capabilities, but a high score—even on an agent-focused benchmark—doesn't actually confirm that an AI can correctly handle diverse, real-world jobs. Patronus AI, a startup founded in 2023 by former Meta AI researchers Anand Kannappan and Rebecca Qian, is helping model creators and companies refine their models to achieve this by constructing simulated digital environments where agent performance can be evaluated.

The San Francisco-based startup appears to be tackling a critical challenge. According to Glenn Solomon, a managing director at Notable Capital, nearly every leading frontier AI lab and numerous emerging startups are now clients, and he describes demand for the company's simulated environments as almost insatiable. Patronus' revenue has surged 15-fold over the past year, drawing significant investor attention. On Thursday, the company announced a $50 million Series B round led by Greenfield Partners, with participation from Notable Capital, Lightspeed, Datadog, and Samsung. This funding brings Patronus' total raised to $70 million.

Patronus employs what it calls "digital world models" to craft replicas of websites and internal systems. Within these environments, agents are rigorously tested after training using reinforcement learning, which systematically rewards successful task completion and penalizes errors. AI labs find great value in these digital simulations because they allow agents to explore different, sometimes unpredictable, scenarios. The company likens its approach to how Waymo trained autonomous vehicles by first building synthetic worlds to test them against rare hazards, like severe weather or a child chasing a ball. The key difference with AI agents is their tendency to take shortcuts, which often leads to incomplete or incorrect task execution. "Patronus is really good at spotting the hacks and making sure they are holding the models accountable," Solomon said.

Currently, Patronus provides its simulated digital worlds for software engineering and finance, but Kannappan notes these are just the beginning. "Today we're very focused on the problems that are verifiable, so the problems that you can immediately check and verify, but there are a ton more areas that are very non-verifiable or very hard to verify," he explained. Just because these processes are verifiable doesn't mean they are straightforward. "We want to be able to actually create the environment in which you can operate an agent that can run for 10 hours or 10 days or 10 weeks," Kannappan said. Regarding competition, Patronus believes its main rivals are the internal teams that AI labs have already established to evaluate agent behavior. While human-data firms like Mercor and Surge assist model makers with reinforcement learning, Patronus differentiates itself by assessing how agents behave without any human involvement.