The first AI worm

hazel · 03-05-2024, 08:59 AM

This worm is only experimental but similar malware could be out in the wild in a couple of years. It infects intelligent email agents.

Quote:

To create the generative AI worm, the researchers turned to a so-called “adversarial self-replicating prompt.” This is a prompt that triggers the generative AI model to output, in its response, another prompt, the researchers say. In short, the AI system is told to produce a set of further instructions in its replies. This is broadly similar to traditional SQL injection and buffer overflow attacks, the researchers say.
To show how the worm can work, the researchers created an email system that could send and receive messages using generative AI, plugging into ChatGPT, Gemini, and open source LLM, LLaVA. They then found two ways to exploit the system—by using a text-based self-replicating prompt and by embedding a self-replicating prompt within an image file.

https://www.wired.com/story/here-come-the-ai-worms/

rokytnji · 03-05-2024, 09:16 AM

So basically It tell you watch out of ai generated images on line while browsing. Watch out for sites using autofill.

I sure can't tell. Tell a LLM to act as a scammer. Gotta look that up also.

Quote:

by using a text-based self-replicating prompt and by embedding a self-replicating prompt within an image file.

Gotta readup on things like jail break.

I guess my best defence so far is I do not willy nilly autofill anything. Sounds dangerous.
Guess browser is attack region.

mjolnir · 03-07-2024, 09:45 AM

Thanks for the link, hazel. It was an interesting read.