Policy & Regulation

OpenAI lets outside safety assessors inside, but critics say it is far from 'employee-like access'

Staff from four safety groups will work as embedded assessors. Researchers say their access is narrow, with no internal Slack and limits on which OpenAI employees they can talk to.

Engineer seen from behind inspecting server racks with a flashlight, no faces, logos or readable text
AI-generated

OpenAI has started a programme that puts staff from four AI safety and research organisations inside the company as “embedded assessors”, but several safety researchers say it gives them much less access than chief executive Sam Altman promised, OPB reports in a story first produced for NPR.

What OpenAI announced

OpenAI announced the programme on Monday. According to OPB, the company says the assessors will work on different parts of AI safety, from stopping models attempting undesirable behaviour to catching that behaviour when it happens. They will have access to some internal models, training processes and some OpenAI employees, and can publish their findings once OpenAI has redacted confidential information.

“Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable,” Altman said at Semafor’s Silicon Valley and the World conference on Monday. Lama Ahmad, OpenAI’s frontier assurances lead, wrote on X that the company was “giving deep access to some of the most sensitive tools and models our researchers use”.

Background

Earlier stories on this thread.

What happened next

Later stories on this thread.

Why critics are unhappy

In September, Altman said independent evaluators would get “employee-like access” to internal systems. Researchers quoted by OPB say the published terms fall short of that.

“The assessors have very narrow scopes to their access, and only two of them seem to really be allowed to talk to employees,” said Sydney Von Arx, founder of the AI safety group Nightingale. She also pointed out that “nobody has access to internal Slack”.

One of the assessors, Apollo Research, set out its own limits: if an OpenAI employee who has not been pre-approved contacts the assessors, they have to bring OpenAI in before a substantive conversation, OPB says.

Tomek Korbak, one of three OpenAI safety staff who were fired earlier, wrote on X that embedded auditing “will only fulfill its aims if the auditors really get employee-level access”, and called Altman’s September post “an awkward overpromise”.

OpenAI’s answer

In a statement to NPR, OpenAI said there are no standards yet for embedded evaluations. “We expect to deepen these relationships over time and share what we learn to help establish best practices for effective embedded assessments across the industry,” spokesperson Gaby Raila said. The company added that assessors can ask to speak to employees who have not been pre-approved, and that there is a process for getting statements from OpenAI about things they learn informally.

Our read

Analysis: the programme’s value will show in what the assessors are allowed to publish. The first redacted reports will be the real test of how much they could see. Anthropic has published its own guidance on designing safety evaluations, so both labs’ approaches can be compared.

This report is not legal, financial, or health advice. See the disclaimer.

Primary sources

  1. OpenAI announces new AI safety program; researchers worry it won't be enough OPB / NPR

Tags