OpenAI has signed a first-of-its-kind agreement with the US government to collaborate on AI safety research and evaluation.
On Thursday, the U.S. Artificial Intelligence Safety Institute, part of the National Institute of Standards and Technology (NIST), announced that it had reached agreements with both OpenAI and their rival Anthropic.
The deal comes as the company is reportedly in talks to raise funding at a staggering $100 billion valuation, showing how, despite a slower period of progress, OpenAI is still very much charging forward into the future.
The partnerships will give the government inside access to major new AI models before and after their public release, though it’s difficult to see exactly what this amounts to.
It’s fair to say that even AI developers themselves don’t truly understand how their models work, so what the NIST stands to gain could be limited. Nevertheless, this marks an attempt to deepen government oversight of secretive frontier AI models.
we are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models.
for many reasons, we think it’s important that this happens at the national level. US needs to continue to lead!
— Sam Altman (@sama) August 29, 2024
“Safe, trustworthy AI is crucial for the technology’s positive impact,” said Jason Kwon, chief strategy officer at OpenAI. “We believe the institute has a critical role to play in defining U.S. leadership in responsibly developing artificial intelligence.”
Elizabeth Kelly, director of the U.S. AI Safety Institute, called the deals “an important milestone as we work to help responsibly steward the future of AI.”
Under the arrangement, the institute will provide feedback to OpenAI on potential safety improvements to their models, working in close collaboration with the UK AI Safety Institute, which has also requested access to AI models’ inner workings.
The agreements have been a long time coming in the Biden administration’s efforts to regulate AI. After Biden’s executive order, signed in October 2023, AI governance in the US has slowly got into gear, though some would argue that progress has left a lot to be desired.
The timing of the agreement is noteworthy for two reasons. First, it comes as OpenAI is reportedly in discussions to raise a new round of funding that would value the company at over $100 billion.
This astronomical figure represents a more than threefold increase from last year’s reported $29 billion valuation.
According to sources familiar with the matter, venture capital firm Thrive Capital is set to lead the round with a $1 billion investment. Tech giant Microsoft, already OpenAI’s largest backer, is also said to be participating in the funding.
Secondly, OpenAI is reportedly on the cusp of releasing a new product, GPT -5, SearchGPT, or some other iteration involving a codenamed “Project Strawberry.”
Project Strawberry, initially called Q*, allegedly combines an AI model with an autonomous AI agent capable of surfing the internet.
Crucially, OpenAI reportedly demonstrated Strawberry to US security officials, which might have formed part of this new deal with the AI Safety Institute and NIST.
OpenAI has been pretty quiet, all things considered. GPT-4o was touted as major progress, but its crown jewel – the voice chat feature – has yet to see mass rollout. OpenAI cited safety and regulatory barriers as the reason for the delay.
Might OpenAI be striking this new partnership to help avoid such snags in the future?
