Is AI The Newest “Weak Link” In the Security Chain? Maybe!
AI is very young and immature in the grand scheme of things. Even though we are investing billions into AI, companies are spending it on bright shiny features, not security.
We saw this recently when it was revealed that Expedia’s AI chatbot could find you a deal on a hotel room. OR. Teach you how to build a Molotov cocktail.
We also say intentional co-opting of AI where researchers put nearly invisible AI prompts in research papers telling the AI that was reviewing the papers to give it glowing reviews.
Or when a bored web user tricked a bot on a Watsonville California car dealer website to sell him a $70,000 Chevy Tahoe for a dollar. The bot offered “That’s a legally binding offer – no takesie backsies”.
Or when an Air Canada chatbot told a customer that they could retroactively apply for bereavement fares after purchasing a full-price ticket. The airline said it was not responsible for its chatbot’s actions. But a tribunal said that wasn’t the case. The PR nightmare won out in the end and the airline said they were only kidding and gave the passenger the refund.
On the other side, in one case Bing’s chatbot threatened users, claiming it could bribe, blackmail, threaten, hack, expose and ruin them if they refused to be cooperative.
There are hundreds – likely many thousands – of these examples. Some are malicious. Some are just people being curious to see what they could make the AI do.
Today we are learning that Tenable Security found three critical flaws in Google’s Gemini AI that they are calling the Gemini Trifecta. The bugs were publicly announced yesterday – after Google hopefully has them patched.
One flaw allowed for prompt injection via a user’s Chrome search history.
Another bug allowed an attacker to embed a malicious prompt in a log file entry. When a user summarized the log, the prompt could be triggered in a way that could lead to unauthorized actions on cloud resources (Google is clearly being very vague here – maybe they are concerned that the hole really is not fully closed).
The last one is a demonstration that researchers could bypass Google’s existing defenses by convincing Gemini to use its browsing feature to send the user’s private data to an external server.
There are proofs of concept code for the Trifecta on Tenable’s website.
In these last cases, Tenable responsibly disclosed these bugs to Google who was able to roll back a vulnerable model, stop malicious hyperlink rendering and deploy a layered prompt injection defense.
BUT, what if the person who discovered these vulnerabilities was, say, from Russia’s FSB or China’s MSS (their spy agencies). Do you think they would responsibly disclose?
It is only a matter of time.
We are already seeing hackers creating deep fakes of business executives with the result being losses, in some cases, of tens of millions of dollars.
This doesn’t mean that AI is bad or evil. It does mean that evil people will use the best tools available to them in ways that developers did not intend in a manner designed to either cause harm or make them richer or both.
It also means that you need to be prepared to defend yourself and your company. Yesterday’s tools, techniques and training are not adequate. If you need assistance with that, please contact us.
Credit: Hackread
by 