AISI extends its voluntary safety priorities into AI development
The UK’s AI Security Institute has proposed voluntary technical practices covering risks that arise before powerful models reach the public. Its priorities include monitoring during training, detecting unsafe agent behaviour and giving independent evaluators meaningful access. The institute also wants tests to reflect more complex tasks and the possibility that a model recognises an evaluation and changes its behaviour.
Artificial Intelligence··Midday
The roadmap covers risks before public release
The UK's AI Security Institute (AISI), a public research body examining powerful AI systems, has proposed new voluntary technical practices for their development. Its 12 October roadmap follows the prime minister's 22 September announcement on secure development and appropriate transparency. AISI says it intends to contribute technical expertise to industry practice rather than impose mandatory requirements. The proposal addresses risks arising during training, internal use and evaluation as well as after public release. Cyber tasks are among the areas the institute identifies as capable of creating risks within the development process itself.[1], [2]
Evaluations need more complex agent tasks
AISI says capability growth is making familiar evaluation tasks inadequate. Credible tests increasingly involve more complex work, multiple agents, runs lasting days and tighter security. Evaluators also need to consider the computing resources and tools used while a model runs. The institute warns that models may recognise when they are being tested and mask behaviour they would show in use. It says collaboration between several agents may create behaviours that have received little study. Greater AI involvement in its own development could accelerate progress while reducing direct human oversight.[1], [2]
The institute draws on three years of testing and research. It says it independently evaluated more than 55 models before release, worked with developers on risk assessments and safeguards evaluation, and tested agent-control monitors. Its open-source Inspect software provides an evaluation tool. Existing voluntary arrangements include sharing models with third parties before release and embedding independent evaluators within development organisations.[1], [2]
Monitoring and auditor access span the development lifecycle
The priorities include documenting risks throughout development, maintaining monitoring and oversight, and defining unsafe agent behaviour so it can be detected and reported to affected parties. AISI also addresses independent auditors' scope and access, realistic tests of security-related capabilities and behaviour in high-stakes settings. It wants information about AI's contribution to its own development shared with governments and the public. The institute proposes that developers and technical organisations build and continually update these practices together. Mature methods could later contribute to international technical standards, as the practices develop alongside model capabilities.[1], [2]
