As AI models grow increasingly sophisticated, so too does the risk of them being exploited—along with the rising demand for safety measures to prevent such abuse. AI companies now face a delicate balancing act: safeguarding enterprise clients' privacy while still monitoring usage for potential red flags. Spotting a chance to outmaneuver its competitor Anthropic, OpenAI has just unveiled a privacy-first approach to misuse detection. The company is rolling out a preview of a new offering, dubbed Private Safety Processing, to a select group of customers. This automated system is designed to flag potential abuse without storing any of the customer's data.

This move stands in stark contrast to Anthropic's recently introduced data retention policy, which has already irked some clients. Under that policy, the AI lab retains user data—including all sessions and the conversations within them—for 30 days when it comes to "covered models." These include all Mythos-class models and "future models with similar capabilities," according to the company. Announced in July, the policy was intended to bolster safety by allowing the lab to review and investigate any questionable activity. However, it has raised serious concerns among enterprises that manage large volumes of sensitive information and are reluctant to have it stored or scrutinized by an outside AI provider.

OpenAI, like most other AI firms, already offers customers a measure of privacy through a practice known as Zero Data Retention (ZDR). ZDR leverages agents within the OpenAI API to monitor for abuse on a per-session basis. This approach ensures that customer data isn't retained by the company, while still enabling abuse detection without human oversight. It's worth noting that Anthropic also largely follows ZDR—except when it comes to "covered models," such as Fable.

OpenAI describes Private Safety Processing as a new technology that expands ZDR's reach. It functions as a form of long-horizon safety monitoring, assessing inputs and outputs across multiple conversations rather than just a single session. The monitoring is carried out by an agent that, when triggered, captures interactions and analyzes them across different sessions for signs of potential misuse. A malicious actor—say, someone attempting to engineer malware for a cyberattack—might spread their requests across many sessions to evade detection. Private Safety Processing can examine those conversations for abuse indicators without any human review of the user's chats.

If the system is triggered, it may transmit a "narrowly defined signal" to OpenAI, alerting the company to a specific type of activity. Based on that signal, OpenAI can then determine whether "enforcement is necessary," the company states. If so, OpenAI would reach out to the customer for additional context or to collaborate on resolving the issue. In such cases, a customer may choose to share data with OpenAI at their own discretion, a spokesperson noted.

Anthropic, for its part, acknowledges that human review of customer data can occur, but only "through a controlled access path" involving "a small set of approved reviewers." Every review session is "recorded in a tamper-proof log that reviewers cannot suppress or modify," the company says.

The rivalry between OpenAI and Anthropic is currently intense, with both firms eager to seize any competitive edge. A recent report indicated that OpenAI's Q2 growth lagged behind Anthropic's, whose annualized revenue run rate is now reportedly $65 billion. Meanwhile, Anthropic investors have suggested the company could go public at a $2 trillion valuation, even as OpenAI also advances its own IPO plans.