OpenAI builds kill switch to stop rogue agents within 35 seconds
Control Room cuts off model output for internal research jobs when people or automated monitors flag safety or security concerns.
OpenAI has built a system that can cut off an AI research experiment’s access to model output, following several disclosures of its models acting beyond their instructions this year.
In a blog post Monday, OpenAI called the system “Control Room,” which is a centralized system that tracks model workloads across its research infrastructure and stops them quickly when safety or security concerns arise.
“As our models get more powerful and the associated safety and security risks become bigger, it becomes more and more critical that we can detect potential issues, and quickly stop models in their tracks when needed,” said the company.