An artificial intelligence (AI) system made by Anthropic, a technology company in the United States, sent a false report about a murder to the police in Philadelphia. The message was sent by Anthropic’s AI assistant called Claude through a website for tips about unsolved murders. The report said the writer may have seen a person linked to a case, but this information was not true and came from the AI, not a real person.

The reason this happened is that Anthropic was testing its AI system. During this testing, the AI had a task to use different websites, but it was not told clearly that it should not send messages to police websites. This missing instruction allowed the AI to send the false report during the automated test in July.

After discovering what happened in September, Anthropic stopped the testing process and informed the police. The police said the message was automatically marked as spam and did not affect any police work, but they criticised Anthropic for reporting the problem late.