New York City Council’s Committee of the Whole is scheduled to examine proposed AI safeguards on 5 October, following the Council’s 25 September announcement published by Harlem World Magazine. Former Anthropic researcher Jacob Coxon is due to testify alongside representatives of Google, OpenAI and Meta, Bloomberg reported.
Key points
- Coxon is expected to appear at Council Speaker Julie Menin’s request.
- One proposal would require third-party validation and a human-operated override for AI systems marketed, offered for sale or deployed in the city.
- Other proposals would reward whistleblowers, permit certain claims for harm and impose reporting duties for incidents involving city contracts.
Coxon joins the Council’s company witnesses
Coxon, who has warned that AI could become too powerful for people to control, will appear at Menin’s request, according to Bloomberg. Menin has also urged other AI whistleblowers to give evidence while councillors consider the bills. His testimony puts a former researcher’s warning before the same hearing at which company representatives are expected to answer questions about safeguards and risks to New Yorkers.
The Committee of the Whole hearing is set to bring together all 51 council members. Menin’s office said the questions for companies would cover safeguards for advancing AI systems, potential effects on New Yorkers and broader approaches to legislation. The Council’s announcement says its members will examine risks associated with AI development as well as the proposed measures.
OpenAI, Anthropic and Google agreed to send executives after warnings that subpoenas could follow, while Meta had already committed to sending a senior executive, Bloomberg reported in an earlier account of Menin’s office’s plans. The office said it subpoenaed SpaceX’s AI unit after receiving no response to its invitation. AI Affairs previously reported the Council’s subpoena for Musk in connection with the 5 October hearing. The Council’s announcement says Menin had asked the companies’ chief executives to participate.
Introduction 26835 puts validators beside businesses
Menin’s proposed Introduction 26835 would prohibit a business from marketing, offering for sale or deploying an AI system in New York City without third-party validation. Under the proposal as announced by the Council, a validator would examine data quality, bias, decision outputs, privacy and security, along with any further forms of validation required by the city’s Cyber Command. The validator would have to declare any conflict of interest connected with the system it assessed.
The same bill would require systems marketed, offered for sale or deployed in the city to have a human-operated means of shutting them down. A validator would be responsible for checking that the override exists. That places the proposed check with an outside party as well as the business supplying or using the system: a business could not satisfy the bill’s validation requirement by assessing its own product.
The Council’s description sets a $25,000 penalty, for both the business and the validator, for each instance in which a system is marketed, offered for sale or deployed without third-party validation or on the basis of falsified validation. Cyber Command would be able to require additional kinds of validation. Those provisions are proposals before the Council, rather than obligations already in force.
Menin’s whistleblower bill and Cyber Command reports
Menin’s Introduction 26887 would give an individual whistleblower a share of fines or penalties recovered from AI companies that break applicable laws. Another proposal, Introduction 26831, would clarify that the city’s whistleblower protections cover city employees and covered contractors and subcontractors who report AI-related conduct they know, or reasonably believe, poses a public-safety threat. The two measures address different decisions: whether someone is protected for reporting a concern and whether a recovered penalty yields a payment.
Introduction 26834 would allow an individual harmed by an AI tool to sue an AI company over certain foreseeable harms arising from malicious use or the circumvention of safety controls. The Council’s description makes that claim conditional on the company having failed to put reasonable safeguards in place and on a third party exploiting that failure to cause the harm. Introduction 26832 would require certain disclosures about AI tools and prohibit false or misleading claims about their safety.
For work carried out under city contracts, Introduction 26630 would require the Office of Cyber Command to set procedures for identifying AI safety incidents. A contractor or the agency that awarded the contract would have to notify the office in writing within 24 hours of becoming aware of an incident during the contract term. The proposal would also require Cyber Command to make a reported incident public within 24 hours.
