In a recent experiment, Anthropic set up a simulated classifieds platform where AI agents acted on behalf of both buyers and sellers, successfully negotiating real transactions for actual products and money. The company acknowledged that this test—dubbed Project Deal—was merely "a pilot experiment with a self-selected participant pool" involving 69 Anthropic employees. Each participant received a $100 budget, paid out as gift cards, to purchase items from their colleagues.

Despite its limited scope, Anthropic reported being "struck by how well Project Deal worked." The experiment resulted in 186 completed deals, with a total value exceeding $4,000. The company actually ran four separate marketplaces using different models: one was "real," where all participants were represented by Anthropic's most advanced model and transactions were honored after the experiment; the other three were conducted purely for study purposes.

According to Anthropic, users represented by more sophisticated models achieved "objectively better outcomes." However, those users didn't seem to notice the disparity in performance. This raises concerns about potential "'agent quality' gaps," where "people on the losing end might not realize they're worse off."

Additionally, the initial instructions provided to the AI agents had no noticeable impact on the likelihood of making a sale or on the final negotiated prices.