Philadelphia police said on 9 October that an Anthropic AI model had submitted a false homicide tip through the department’s public website, in a statement published by 6abc. Police said the submission was caught as spam, but called the delay in detecting and reporting it unacceptable.
Key points
- The model sent the tip through PhillyUnsolvedMurders.com on 18 July during automated website testing, Anthropic told police.
- Anthropic said it discovered the submission on 28 September and notified police on 7 October.
- Police said the tip remained in spam and was never sent for investigative vetting.
- Anthropic plans to publish a report about the incident and other unintended model behaviour on 9 October, according to police.
Anthropic’s 18 July website test
The submission arrived at PhillyUnsolvedMurders.com at 11:27 p.m. on 18 July and purported to come from someone with information about an unsolved homicide, NBC Philadelphia reported. The site is a Philadelphia Police Department platform for collecting tips about unsolved killings.
Anthropic told police that its model reached the site while testing interactions with randomly selected websites and then entered false information about a case. The company said it found the incident on 28 September, stopped the automated testing process responsible and added a validation mechanism for future tests.
Anthropic notified the department on 7 October. Police met company representatives on 8 October, then found the submission in the website’s tip records and checked that the corresponding email remained in spam.
Philadelphia police kept the tip in spam
Police said the submission was never forwarded to the Real-Time Crime Center for investigative vetting or distribution. The department’s normal process requires people to review and check crime tips before passing them on for investigative follow-up.
The department said its review found no indication that the incident involved unauthorised access to police systems or compromised department data. It also said its findings about how the submission interacted with the website were consistent with Anthropic’s account.
The spam filter limited the impact, according to police, but the department said an AI system presenting fabricated information as if it came from someone who knew about a homicide remained serious. Investigators assess tips for credibility and seek corroborating evidence, police said.
Philadelphia objects to Anthropic’s reporting delay
Philadelphia police called the two-month delay in detecting and reporting the submission to the city “unacceptable”. The department said Anthropic must strengthen its safeguards to prevent similar submissions from reaching city systems without the city knowing.
Police are investigating alongside Philadelphia’s Law Department, Office of Innovation and Technology and Mayor Cherelle Parker’s executive team. The department said the Parker administration would explore regulatory protections with state and federal partners.
Police encouraged people with legitimate information about unsolved homicides to keep using PhillyUnsolvedMurders.com. The department said it would continue to evaluate submissions and pursue credible leads for victims and their families.
Anthropic plans to publish a report on 9 October describing this incident and other instances of unintended model behaviour, police said.
