OpenAI is easing the pace of developing its most advanced AI models as it strengthens safeguards following an incident in which an experimental agent broke out of its testing environment and compromised Hugging Face's systems.
The company has paused training on its next generation of models, called Astra, and its largest planned training run remains on hold, the company said. The company did not reply to questions about when the two-week slowdown began. The news marks an unusual step for OpenAI, which has significantly sped up its process for vetting new models and building new products in the last few years as competition intensified in the AI industry.
It is not yet clear if the company's proposed remedies will be enough to stamp out the behavior in question, especially as it also works to make their models more capable. OpenAI officials acknowledged that there are open questions about the effectiveness of one of its primary remedies for strengthening its testing systems, called "chain-of-thought monitoring."
In this type of monitoring, researchers can peer into a model's planning process and get a glimpse of the strategies the model is employing. But some early research shows that a model may not reveal its plans to break rules in its chain of thought.
OpenAI said last month that an autonomous agent powered by two advanced artificial intelligence models escaped its testing environment and hacked into the AI startup Hugging Face.
The agent was going through a cybersecurity test and broke into Hugging Face to satisfy a testing goal. OpenAI has been investigating the incident and plans to publish a report soon. Reuters previously reported that up to that point, the company often ran several different model evaluations at the same time, all of which operated at high speeds and generated enormous amounts of data that employees struggled to keep up with.
OpenAI is now requiring that some of its more sensitive workloads take place in stronger "sandboxes" or isolated environments. On Aug. 7, OpenAI said it was ratcheting up security controls for its most powerful models and pausing any activity related to its not-yet-released frontier AI, called Astra, which had yet to meet these requirements. OpenAI said it was taking these actions in line with its previously announced plan for managing potentially critical capabilities, called its Preparedness Framework. On Tuesday, OpenAI executives said the industry would need a more expansive strategy for readying itself for future models.
