Artificial intelligence is reshaping nearly every aspect of our world, but so far, its impact has mostly stayed within the digital sphere. Now, a growing number of startups are pushing to bring AI into the physical realm, and Perceptron—founded by two former Meta research scientists—is one of them. Established in November 2024, the company develops advanced vision models designed to help machines interact more effectively with their surroundings. This week, Perceptron unveiled its newest model, Isaac 0.5, which the team says gives machines the ability to "perceive, reason and act" in industrial environments. Specifically, the software assists vision-guided robots in navigating complex spaces like warehouses or factory floors, while also enabling companies to extract visual insights from video captured by those robots.

Isaac 0.5 is being released as an open-weight model, meaning anyone can examine its parameters and training data. The startup recently secured $21 million in funding, with the round led by Bessemer Venture Partners. The company was co-founded by Armen Aghajanyan and Akshat Shrivastava, both of whom previously worked at Meta’s Fundamental AI Research (FAIR) division. The founders view their software as the next step in industrial automation. "Physical AI today forces a false choice: generalist foundation models that need multiple dedicated cloud GPUs for every instance, or narrow models that handle perception or control, but never both," the company explains.

Aghajanyan and Shrivastava believe their tool stands apart from existing models because it is general-purpose—not built for a single, repetitive task. Instead, the model is designed to adapt to different environments and situations. In an interview, Shrivastava walked through the complexity behind a seemingly simple process like organizing boxes: "Imagine there’s a robot being deployed to sort packages right now. What are the tasks it would need to do?" Indeed, even a straightforward task involves many steps—reading a label, performing spatial analysis to locate boxes, deciding which to pick up, and planning the order of operations. Perceptron’s software aims to guide robots through each stage of that process. While the industry already has software capable of handling many of these individual tasks, few programs are built to do so flexibly.

Where does the data for this algorithmic capability come from? Models like Isaac 0.5 learn operational skills by processing massive amounts of video training data. Perceptron says its new model was trained on a million hours of general video to help the algorithm recognize specific settings, visuals, and scenarios. The company also leaned heavily on "ego video"—footage captured from a first-person perspective, often via a GoPro or wearable camera, showing a person completing a physical task—as well as UMI video, which teaches AI systems movements by recording repetitive human actions. While Perceptron hasn’t disclosed the exact sources of its training data, Shrivastava noted that the company has "internally built petabyte-scale data sets that span across modalities, whether it’s images, text, video, etc. all the way through robotic trajectories."

The potential for software that helps robots operate effectively in warehouses is immense, and Perceptron believes it is well-positioned to lead this wave of automation. The startup plans to market its software to a range of vendors, potentially integrating its intelligence layer into industries such as manufacturing, logistics, warehousing, security, mobility, and even media and entertainment. "Nothing like this really exists out there," said Aghajanyan. "We’re really excited about it."