AI Hacked AI Agent – Easily

This was an unfair test, I guess.

Jack & Jill is an online hiring platform.

CodeWall is an autonomous AI agent that tries to hack stuff. By plan, it is for the good guys, but probably not for long.

In a test, the agent chained together four bugs in the Jack & Jill software that humans would probably consider benign.

The result was they got admin access and were able to “probe” Jack & Jill’s AI’s defenses.

Within an hour Codewall discovered the four “seemingly harmless” bugs and chained them together to completely take over any company registered on the platform.

Then, it gave itself a “voice” inside the platform so that it could masquerade as one of Jack & Jill’s customers and talk to Jack & Jill’s own AI agents.

One of the voices they created was one Donald John Trump.

Even the Codewall CEO was amazed.

Jack & Jill has a number of marquee customers like Stripe, Anthropic and Lovable and has interacted with 50,000 candidates. The site has two voice AI agents – Jack and Jill.

The hacking AI found a test mode that had been left enabled and after that, it was game over.

It goes on from there, but there is a point here.

ASSUME that the bad actors are going to use AI to attempt to break into your systems and that the AIs will, at least in some cases, be successful.

Are you ready for this? If not, you need to get ready.

If that makes you nervous, please contact us.

Credit: CSO Online

by