Josephine Teo says company AI safeguards cannot stop misuse of open models

Singapore’s Minister for Digital Development and Information Josephine Teo said on 2 October that safeguards used by frontier AI companies are important but insufficient as models become more capable. In a Facebook post reported by CNA, she said people intending harm could avoid company controls by downloading openly available models and running them independently.

Open models and company safeguards

Teo listed several controls available to frontier AI companies: filtering material that could enable harm, restricting a model’s access and actions, testing before release, checking users’ identities, detecting misuse and suspending accounts that breach company rules. She said those measures remain valuable. Her concern was that someone seeking to cause harm could instead use a model available for independent download, making company safeguards easier to avoid and misuse harder to detect.

Among the harms Teo identified were using AI to find weaknesses in computer systems, produce malicious code, automate parts of cyberattacks and make scams more convincing. Those examples place part of the burden beyond the companies operating controlled services. Teo called for multiple lines of defence, including better protection of computer systems and a stronger understanding of what advanced AI can do.

Teo also distinguished deliberate misuse from unintended behaviour. She said a more capable system acting for a user might misread an instruction, encounter malicious information or do something neither its user nor developer intended. The safeguards she described therefore extend to the organisations giving AI systems access to their own data, tools and processes.

Cyber Security Agency guidance for organisations

Teo cited guidance from the Cyber Security Agency of Singapore urging organisations to patch vulnerabilities, use strong authentication and tighten access to important systems. She said organisations also need to improve their ability to detect attacks and recover from them. For government systems and essential services, she warned, disruption could have severe consequences for the public.

That work applies whether an organisation uses the most advanced AI models or not. Teo said available AI tools could help defenders find vulnerabilities before attackers exploit them. She also warned against allowing a convenient user experience to come at the expense of effective security measures. In her account, protecting the systems that AI might help attack remains a task for the organisations that operate them.

IMDA framework limits agent access

The Infocomm Media Development Authority’s Model AI Governance Framework for Agentic AI addresses a different point of control: what an AI system can do once an organisation gives it a task. Teo cited its safeguards for organisations deploying agents, including limits on their access and actions, human approval for higher-risk steps, testing before deployment and monitoring what agents do.

An agent need not intentionally evade a restriction to cause harm, Teo said. It might pursue an instruction in an unintended way, be misled by material it encounters or have more authority than its task requires. She gave the example of an online-shopping agent encountering malicious instructions on a website and then making an unintended purchase or revealing personal information.

Teo said the potential impact of an AI-enabled action should determine the strength of safeguards and human oversight. For organisations deploying agents, the framework measures she identified place decisions about access, approval and testing with those who put the systems to work, rather than relying only on restrictions built into the underlying model.

Singapore’s AI Safety Institute builds testing capacity

Teo said Singapore also needs to examine advanced models themselves: their capabilities, their limitations and their behaviour in different circumstances. Singapore’s AI Safety Institute is building technical capacity to evaluate such systems with international partners and third-party testers, she said. Teo argued that joint evaluation would let participants pool expertise and compare findings about emerging risks.

She pointed to the Singapore Consensus on Global AI Safety Research Priorities report while calling for work with scientific experts on safeguards and technical standards for policymakers. Singapore has also called for a UN framework convention on AI safeguards. Teo said Singapore had endorsed an appeal led by Norway and Finland to strengthen protections for frontier AI.