Software has Bugs and AI is Software – Just at a Different Scale

New reports are coming out about how to jailbreak Microsoft’s GitHub AI assistant to get it to do, well, bad things. Not a big surprise, of course.

Researchers have discovered two new ways to manipulate GitHub’s artificial intelligence (AI) coding assistant, Copilot, enabling the ability to bypass security restrictions and subscription fees, train malicious models, and more.

While researchers say the bugs are vulnerabilities, Microsoft says they are “off-topic chat responses” or “an abuse issue”. Probably technically correct, but that characterization doesn’t solve the problem.

It does allow Microsoft to do what it often does and say this is not a bug so we are not going to fix it.

They did put out some PR fluff about continuing to improve on safety … blah, blah, blah.

As users (and hackers) continue to learn about AI tools, expect a lot more of these. It always happens when VCs rush to market to create a lead or make some money.

The researchers also figured out a way to steal CoPilot’s authorization token to the back end and access Open AI’s API using Microsoft’s bank account. That attack is actually pretty cool.

Credit: Dark Reading

On the other side of the ocean, China’s DeepSeek doesn’t have any better security than many US companies.

During a routine security assessment, researchers at Wiz discovered that DeepSeek had an unprotected ClickHouse database, open to anyone with internet access. This database wasn’t just visible; it allowed full control over stored data, meaning an attacker could manipulate or extract critical information without restriction.

As a result of this find, the researchers were about to query the Chinese system with no credentials.

What they found were:

• API Keys (basically passwords used by programmers to access the system)
• Chat histories
• Backend server details
• System operational metadata

The data includes detailed logs of internal system activity.

In one sense, this hack is good. Rather than China stealing our IP, this allows us to steal their IP. And if you think that our wizzes didn’t do that, I have a bridge in Brooklyn that I would like to offer you.

Of course, the chat histories include anything that YOU asked it along with the answers, so not only was China’s IP exposed but your potentially sensitive data was also exposed.

It looks like China shares at least one business model with Meta – move fast and break things.

Our only comment is that you should understand the risks and if you do and decide to proceed, then more power to you.

If you would like to talk to our Chief AI Officer about this, please contact us.

Credit: Hackread

by