OpenAI sets up security team Preparedness: assess AI risks and prevent abuse by outsiders

CTOnews.com, October 27, OpenAI announced today that it is setting up a Preparedness team led by Aleksander Madry, director of the deployable Machine Learning Center at the Massachusetts Institute of Technology (MIT), to assess the possible catastrophic risks posed by General AI (Artificial General Intelligence,AGI).

▲ source OpenAIOpenAI said that although the future AI model has the potential to benefit all mankind, it may also bring serious risks, so it is necessary to establish a strong framework to monitor, evaluate, predict and prevent these AI models.

It is reported that OpenAI has set up a Preparedness team, which will conduct capability assessment and red team testing on various AI models to track, predict and prevent all kinds of catastrophic risks.

CTOnews.com found that the related risks claimed by OpenAI mainly include the following three main categories:

Persuading human ability: whether the output of AI will affect human behavior

Improper content generation: whether AI will generate hazardous content such as chemical / biological / radiological / nuclear (CBRN)

Autonomous replication and adaptation (Autonomous Replication and Adaptation,ARA): whether AI will get out of human control in self-iteration

OpenAI has begun to recruit people from different backgrounds to join the Preparedness team, while launching the AI Preparedness Challenge, which encourages contestants to "crack" Whisper, Voice, GPT-4V, DALLE3 and other models in the name of hackers, and the top 10 will receive a $25000 quota of API usage provided by OpenAI (CTOnews.com Note: about RMB 183000 currently).