How to hack Copilot AI: ask it how to hack it
Microsoft sells Copilot as your helpful robot servant for your organisation’s data management. Search it, analyse it, everything you could want at work! Except accuracy. And security. Our friends at Varonis found yet another new use case for Copilot in speeding up their security work. [Varonis]…
Microsoft sells Copilot as your helpful robot servant for your organisation’s data management. Search it, analyse it, everything you could want at work! Except accuracy. And security. Our friends at Varonis found yet another new use case for Copilot in speeding up their security work. [Varonis] Varonis asked Copilot how they could get it to fire a prompt from a webpage address without the user having to do anything beyond going to the page. Copilot refused to answer, because that’s against its guard rails. So Varonis asked Copilot more questions about how its guard rails work: Copilot then disclosed an undocumented URL parameter — unprompted, mid-refusal — including its historical behavior and every protection put in place to disable it. Varonis just had to add “autorun=1” to the address. If you can send a known user a link that then runs a prompt with no further interaction, that’s game over. The chatbot can do anything the user has the power to do, and you can make Copilot email company data out to you. Varonis told Microsoft about the attack in December last year. It’s been exploitable for eight months. Microsoft say they finally patched it on Tuesday 18 August. [Microsoft] The lesson is that chatbots are not securable, and that’s why letting a chatbot do things is only going to blow up on you. This hole is closed, but it took Microsoft eight months. I predict a new Copilot hole any moment — because this machine is built to go wrong. Video — PodcastSource: Pivot to AI — Published — Category: Business