Representative Ro Khanna said on 28 September that he would introduce the Human Control Over AI Act, a bill that would prohibit certain forms of recursive AI self-improvement until federal safeguards and agency approval are in place, CNBC reported after reviewing a summary of the proposal.
Key points
- The proposed restriction covers recursive self-improvement and autonomous changes to a model’s core objectives, containment or shutdown controls.
- A new federal agency would license frontier-model training and deployment and receive reports from auditors embedded in labs.
- The bill would impose criminal penalties for disabling safeguards or knowingly deploying an unauthorised system, and require liability insurance for model releases.
The Human Control Over AI Act’s conditional ban
The proposed ban would apply to models that recursively improve themselves or autonomously alter their own core objectives. It would also cover autonomous changes to containment and shutdown controls. Under the summary reviewed by CNBC, those activities would remain prohibited until federal guardrails exist and a federal agency approves them. The approval requirement would put the decision about whether such work can proceed with an authority outside the laboratory.
Khanna described three concerns in his interview with CNBC: extinction, loss of control and misuse. “There’s a safety risk of loss of control, and then there’s a misuse risk, and we need to take both seriously,” he said. The proposed prohibition addresses particular model behaviour and changes to controls. Other provisions of the bill would govern how frontier models are tested, trained and released.
Khanna said he had drawn on conversations with Model Evaluation and Threat Research, the Machine Intelligence Research Institute and Palisade Research rather than requests from executives at large AI companies. He said he wanted the safety community to drive the debate instead of frontier-lab executives. Those are Khanna’s account of the bill’s origins and his stated preference about who should shape it.
A federal agency for frontier labs and chips
The bill would establish an agency focused on models from OpenAI, Anthropic, Google DeepMind and xAI. It would set safety regulations and run a licensing system for model training and deployment. The agency would also oversee frontier-model audits and set security standards for continuous testing and human control. Its role would therefore extend beyond approving the activities covered by the recursive-improvement ban.
Independent auditors would be embedded at every frontier lab and report directly to the agency. That reporting line matters alongside the proposed licence: the body deciding whether training or deployment may proceed would also receive assessments from people working inside the labs. The bill would place those auditors within the organisations whose models they examine, while directing their reports to a federal authority.
The agency would set standards for sandbox testing, for testing environments isolated from the internet, and for kill switches. Its remit would include controls intended to keep models within laboratory settings. These provisions concern both the conditions under which a model is tested and the means available to contain or stop it.
The proposed agency would also regulate advanced chips used by frontier labs, including mechanisms to monitor or control their use. The proposal would have the executive branch seek binding deals and selective controls on overseas shipments to deter China and other adversaries from developing dangerous AI systems. AI Affairs has previously reported on AI company chiefs’ calls for UN controls and US opposition to global oversight.
Penalties and insurance for AI companies
Employees of AI companies would face criminal penalties under the proposal if they disabled safeguards, kill switches, logging or containment systems, or knowingly deployed an unauthorised system. The criminal-penalty provision applies to employees who disable safety systems or knowingly deploy an unauthorised system. A separate licensing system would govern approval for model training and deployment. It would make the operation of safety controls a matter of potential criminal liability for employees who interfere with them.
The proposal would also criminalise “crimes against humanity” in cases where deployment of an AI model results in the destruction of civilian populations. AI companies would have to hold extensive liability insurance to release their models. Together, the proposed penalties and insurance requirement would place consequences on both employees’ conduct and companies’ decisions to put models into use.
The FRONTIER Act and the House timetable
Khanna’s proposal joins the bipartisan FRONTIER Act from Representatives Jay Obernolte and Lori Trahan. That measure would also put independent auditors in frontier labs. It would allow the government to shut down models with the potential for catastrophic risk, whereas Khanna’s proposed ban would apply to specified self-improving or self-modifying activities until guardrails and approval are in place.
CNBC reported that a vote on any House bill is not expected before the midterm election.
