The robotics world is undergoing a major shift as generative AI models increasingly take control of robotic systems. However, this transition introduces a significant challenge: unlike traditional algorithms, generative AI architectures don't behave predictably. This raises a critical question—how can you guarantee that your newly built humanoid robot will operate safely?

Dr. Ding Zhao, director of the Safe AI lab at Carnegie Mellon University, has dedicated nearly his entire career to tackling this exact problem. Together with seasoned startup executive Kyle Wong and machine learning engineer Simo Rachidi, he has launched a new company called Safeworld to address it. As Zhao explains, the safety challenge involves two intertwined components. The first is highly sophisticated probabilistic evaluations for generative AI—essentially, how do you underwrite the risk of a system that operates on probabilities rather than certainties? The second, and equally difficult, piece is establishing trust. Both elements are essential before any robot can be deployed.

EMBED_PLACEHOLDER_0

Safeworld is stepping out of stealth mode today, announcing a seed round exceeding $12 million. The funding was led by Shine Capital and a16z Speedrun, with participation from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel. Jonathan Lai, a partner at a16z Speedrun, emphasized the urgency of the moment: the right time to establish industry safety standards is now, while robots are still being designed and deployed. Once robots are in homes, colliding with children and causing safety incidents, it will be far too late.

Safeworld's core focus is evaluating robotic control systems within simulations populated by realistic human models. This challenge mirrors what companies like Tesla and Wayve face when ensuring their vehicles can handle unexpected road incidents. However, according to Zhao, the difficulty is amplified for robots because they operate in unstructured environments, and every facility they work in will have its own distinct safety standards. Wong illustrates this with a common scenario: imagine a blind corner in a factory. What speed or stopping distance is needed to guarantee the robot won't collide with a person? And if that person is carrying boxes, will the robot even detect them?

To address these questions, Safeworld creates a digital replica of that corner using simulation platforms like Genesis or MuJoCo. It then inserts a simulation of the robot being evaluated, driven by its actual software, and runs thousands of scenarios where simulated humans encounter the robot. According to Zhao, this is more challenging than it appears because human behavior is inherently unpredictable. Wong adds that tripping and falling is another scenario they test extensively in simulation—otherwise, you'd have to physically trip and fall in front of the robot repeatedly, which is impractical.

There are clear overlaps between Safeworld's platform and the internal tools robot developers already use. However, the founders believe that beyond their specialized expertise, robot-makers will want an independent third party to validate their work—if only to enable sharing safety information across competing companies. Zhao warns that many people underestimate how difficult some of these edge cases will be to solve. The concern isn't the robot performing in a controlled demo or in isolation; it's the robot deployed at scale, interacting with people who may have never operated a robot before.

Vishal Dugar, CTO of Gritt Robotics, is building the AI brain for robots that currently assist workers installing photovoltaic panels at industrial-scale solar farms, with ambitions to take on more complex construction tasks. His company is partnering with Safeworld as they develop their safety simulations. Dugar explains that the hardest part of their systems is that formally proving safety through mathematical equations is extremely difficult—it simply has to be verified empirically.

Gritt's robots work alongside human laborers, making it absolutely essential that robotic arms don't strike them. Verifying this in practice requires accounting for an enormous range of potential scenarios. As Dugar points out, humans come in countless forms and configurations. They might be kneeling, standing, tripping, falling, crouching, or running. The robot must respond appropriately to all these possible behaviors, not to mention the wide variations in human appearance—clothing, size, shape, height, skin color, and more.

Both Safeworld and the application of generative AI in robotics are still in their early stages. The company is still determining the optimal business model for its product—whether to offer a platform for external users or take a services-based approach. Nevertheless, the team is confident they are tackling the right problem. Zhao predicts that Safeworld will likely become the first profitable company in this space, because anyone who wants to deploy robots will need to pay them to handle safety.