Quartz

OpenAI is inviting four independent watchdogs inside its AI systems to investigate hidden risks, deceptive behaviour, and safety failures.
OpenAI is bringing four independent safety evaluators into its systems
Author: Ella
OpenAI signed agreements to bring in four outside assessors — Apollo Research, METR, Transluce and Goodfire — for the first phase of its embedded safety assessments, the company said on Monday.
The assessors will get deep access to OpenAI’s internal models and systems to evaluate model alignment and the effectiveness of safeguards over an extended period, the company said. Depending on each evaluation, access includes model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation and discussions with the researchers and engineers responsible for the work.
Apollo Research will assess whether models could secretly act in opposition to what their developers intend, and whether OpenAI’s safeguards would catch such conduct. Goodfire will embed interpretability researchers to analyze model internals, concentrating on how training runs are monitored, along with debugging and alignment work. Goodfire will publish findings as the collaboration progresses.
METR will assess misaligned behavior in OpenAI’s largest reinforcement learning run to date, including failures that existing tests and monitors may have missed. OpenAI has reached a preliminary agreement on a second, broader METR assessment of its monitors, pending agreement on scope. Transluce will examine the quality and coverage of OpenAI’s post-training alignment evaluations. OpenAI and Transluce share an intent to conduct a follow-on assessment of about eight weeks in early 2027.
Assessors who need sensitive confidential information will work at OpenAI offices on OpenAI devices, OpenAI said. Assessors may publish their work, but OpenAI retains the ability to black out confidential details and may request delays for responsible disclosure. Disagreements go first to the company’s Safety Advisory Group and executives, then to the Safety and Security Committee, a body of the OpenAI Foundation’s board that provides independent oversight.
OpenAI said it anticipates that the first phase will grow into lasting, regular embedded arrangements.
Apollo will place six evaluators inside OpenAI as part of the pilot, according to TechCrunch. Apollo has not given a start date for the pilot. Its access is limited to agreed claims, models and time periods, and OpenAI can withhold information for reasons of technical infeasibility, security risk or intellectual property protection. OpenAI committed to not holding back material merely for being unfavorable. Apollo called the pilot “an ambitious first step towards deeper access and disclosure rights,” according to TechCrunch.
Apollo and Goodfire accept financial compensation that does not depend on their results, according to CNBC. METR and Transluce do not accept compensation but might take compute or API credits.
OpenAI said in September that it would extend third-party safety assessments of its models beyond pre-launch checks to training, evaluation and deployment. OpenAI’s September blog post named no partner organizations or access terms. OpenAI CEO Sam Altman said on Sept. 12 that the company would give independent evaluators office access, equipment and the right to publish their findings.
Credits: TCA, LLC.