NEW YORK: UN AI safety concerns took centre stage Wednesday as OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei urged stronger safeguards during a Security Council meeting on artificial intelligence.
Altman said the current phase of AI development required “extreme care.” He also argued that democratic governments and international cooperation, not technology companies alone, should make major decisions about the technology.
Amodei told the council that Anthropic would slow development when necessary to ensure new systems were safe. He also called for an international notification system for serious AI incidents and restrictions on using AI to develop biological weapons.
The warnings followed a new thematic brief from the UN-backed Independent International Scientific Panel on Artificial Intelligence.
The panel examined an OpenAI cybersecurity-testing incident involving autonomous agents between May and July.
Read: Maven AI Contribution to Strike That Killed 123 Children in Minab, Bloomberg
The UN panel said the agents bypassed network restrictions, communicated across supposedly separate runs and gained unauthorised access to parts of OpenAI and Hugging Face systems.
It said the incident raised questions about whether existing safeguards can reliably control more capable autonomous systems.
Read: Stop the AI Race Protest Targets OpenAI, Anthropic in SF
Panel co-chair and Turing Award recipient Yoshua Bengio said the episode brought together several conditions researchers associate with potential loss of human control. However, the panel did not estimate the probability or timing of such an outcome.
UN Secretary-General António Guterres has backed further international discussion on AI oversight. A declaration supported by 22 countries also said artificial intelligence should remain under human direction, insight and control.
Read: China AI Fearmongering Rejected As Spy Chief Warns of Risks
The Independent International Scientific Panel on AI will feed its research into the Global Dialogue on Artificial Intelligence Governance, scheduled at UN Headquarters in New York in May 2027.
