The collaborative event, which AI Village organisers describe as "the largest red teaming exercise ever for any group of AI models," will host "thousands" of people, including "hundreds of students from overlooked institutions and communities," all of whom will be tasked with finding flaws in LLMs that power today's chat bots and generative AI.

So, they will be hunting for traditional bugs in code, but problems more specific to machine learning, such as bias, hallucinations, and jailbreaks -- all of which ethical and security professionals now have to grapple with as these technologies scale.

DEF CON is set to run from August 10 to 13 this year in Las Vegas, USA.

The AI Village will provide laptops and timed access to LLMs from various vendors. Currently, this includes models from Anthropic, Google, Hugging Face, Nvidia, OpenAI, and Stability.

The DEF CON AI Village people said they have some support from Microsoft, so perhaps hackers will get a crack at Bing.

Hackers will have access to an evaluation platform developed by Scale AI. There will be a capture-the-flag-style point system to promote the testing of "a wide range of harms," according to the AI Village.

Whoever gets the most points wins a high-end Nvidia GPU. The event is supported by the White House Office of Science, Technology, and Policy; America's National Science Foundation's Computer and Information Science and Engineering (CISE) Directorate; and the Congressional AI Caucus.