Google has hired a team of contractors to help assess, rate, and improve the responses generated by its AI model, Gemini. The contractors have been shown responses generated by Gemini and Claude, and they then have 30 minutes to rate the response based on factors like truth and verbosity and establish which one is better.
They were unaware that the chatbot they were assessing Gemini’s responses against was Claude until they started repeatedly seeing “I am Claude, created by Anthropic” in a lot of the responses they were assessing. They also noticed that Claude’s safety settings were better than Gemini’s, as it would often refuse to answer queries it considered to be unsafe, whereas Gemini would, leaving the contractors to flag its responses as a safety violation.
Anthropic’s service agreement states that users can’t use its AI model to build or train competing models without permission. Google hasn’t clarified whether it got permission from Anthropic but did say that “any suggestion that we have used Anthropic models to train Gemini is inaccurate.”