AI firms can’t be trusted to voluntarily share risk information

Confidentiality agreements

The letter – signed by both anonymous and named employees – added that “broad confidentiality agreements” are blocking them from voicing concerns, “except to the very companies that may be failing to address these issues”.

“Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated,” they added. “Some of us reasonably fear various forms of retaliation, given the history of such cases across the industry. We are not the first to encounter or speak about these issues.”

The letter ends with calls for greater transparency and accountability from the companies, including that they will not enter into or enforce any agreement that prohibits criticism of risk-related concerns; facilitate an anonymous process for employees to raise concerns; and support internal cultures of open criticism that allows both current and former employees to raise concerns with a wide range of groups.

It also calls on the companies to not retaliate against current and former employees who publicly share risk-related confidential information after other processes have failed: “We accept that any effort to report risk-related concerns should avoid releasing confidential information unnecessarily. Therefore, once an adequate process for anonymously raising concerns to the company’s board, to regulators, and to an appropriate independent organisation with relevant expertise exists, we accept that concerns should be raised through such a process initially.

“However, as long as such a process does not exist, current and former employees should retain their freedom to report their concerns to the public.”

The letter was also signed by prominent AI experts Stuart Russell, Geoffrey Hinton and Yoshua Bengio; the latter of whom was selected by 27 countries during the first global AI summit to lead the first-ever frontier AI State of the science report assessing existing research on the risks and capabilities of the technology.

Commenting on the safety commitments made by the 16 AI firms at the time, Bengio said that while he is pleased to see so many leading AI companies sign up – and particularly welcomes their commitments to halt models where they present extreme risks – they will need to be backed up by more formal regulatory measures down the line. “This voluntary commitment will obviously have to be accompanied by other regulatory measures, but it nonetheless marks an important step forward in establishing an international governance regime to promote AI safety,” he said.

Computer Weekly contacted the Department for Science, Innovation and Technology about Bengio’s comments on the need for mandatory measures, but received no on-the-record response.

While the UK government has repeatedly said it will not legislate on AI for fear of stifling innovation, it outlined its “initial thinking” on the introduction of binding requirements for select companies developing the most powerful AI systems in February 2024, noting that any voluntary measures for AI companies would likely be “incommensurate to the risk” presented by the most advanced capabilities.

Responding to the letter, an OpenAI spokesperson said: “We’re proud of our track record providing the most capable and safest AI systems and believe in our scientific approach to addressing risk. We agree that rigorous debate is crucial given the significance of this technology, and we’ll continue to engage with governments, civil society and other communities around the world.

“This is also why we have avenues for employees to express their concerns including an anonymous integrity hotline and a Safety and Security Committee led by members of our board and safety leaders from the company.”

The spokesperson added that creating safe and reliable systems is critical to OpenAI’s business model, and that 92% of the Fortune 500 are using its tools – something they say would not be happening if the models were not safe or capable.

They further added that OpenAI was among the first in the industry to call for government regulation of AI, and that both current and former employees have many avenues to share their views.

They said this includes the ability raise their thoughts through leadership office hours, an anonymous integrity hotline, Q&A sessions with the board and ongoing communications with their managers.

However, they also said it was important to protect the confidentiality of the technology being built, including from unauthorised disclosures that could confer benefits on malicious actors. They specifically cited countries that seek to overtake the US’s leading edge in AI technology.

Computer Weekly also contacted DeepMind and Anthropic but received no response.