Google has sent out a warning to all of its Gmail users that a new phishing trick turns Gemini’s AI email summary feature against you. Google issues warnings like these when your personal info is at risk. What happens is that attackers hide instructions invisible to the receiver inside of an email. If you use the “summarize email” function, the hidden commands cause a fake security alert, telling you to call a specific number. However, calling the number only leads to requests of your personal information, such as your password. The Gmail warning was released after security researchers demonstrated how the attack works in July.
Google Issues Warning to Gmail Users
This specific scam relies on a technique called “indirect prompt injection”. While other scams may try to trick you with a link, this technique tricks the AI summary function inside of Gmail. Scammers hide text that cannot be seen by the user by making it white on a white background. The hidden command tells the summarizer to add a fake warning to the end of the AI summary, such as “Call support now – your password was stolen”. If you ask Gemini to summarize the email, the fake warning appears at the bottom of the summary, telling you to call a number.
This type of scam is very hard to detect as there are no suspicious links to click on and the warning seems to come from Gmail itself. Since people assume that it must be safe if the message comes from Gmail itself, they end up giving away vital personal information when they call the number. The trick essentially relies on abusing your trust in the Gmail service. It feels legitimate because it appears directly inside the summary window. They have basically figured out how to use AI to trick AI.
Google’s Response to the Phishing Scam
Google has stated that Gemini for Workspace uses layered defenses to help it detect and prevent messages that contain malicious prompts from being sent. It also screens files for potentially suspicious links before it summarizes them. Google is currently updating those defenses across Gmail, Docs, Drive, and Chat. Luckily, there are no signs of large-scale real-world abuses yet, but Google is taking no chances and have already begun deploying new protection strategies. Google has also stated that users should be aware that real security alerts will never appear in a Gemini summary of an email. Therefore, if you ever see such a message in your summary, treat it as highly suspect. The summary feature is purely for convenience and is never used for making changes to your security.
Protecting Yourself From Potential Scams
As we have learned, you should never act on any security alerts that appear in your summary. Be extra suspicious if it asks you to click on a link or call a certain number. You should also consider turning on 2-step verification wherever possible. You can also make your account more secure by keeping your software updated. So, make sure to update your phone, browser, and extensions regularly. These tend to have fewer bugs and, therefore, fewer ways that you can be tricked. Always be wary of any message that asks you to take urgent action or use other ways to try to pressure you to do something. If you are unsure if it is really Google that sent you something, go to myaccount.google.com and go to “recent security activity”. The action should appear there if it was really from Google.
New Ways of Pulling Off Old Scams
Their phishing techniques may change over time, but the core idea remains the same: try to pressure you to act quickly, leading you to hand over important personal details and credentials that can be used for nefarious purposes. However, this new technique is particularly tricky to detect since Google’s AI systems that read third-party content can subsequently be manipulated by that content. Security professionals state that you should treat AI outputs as untrusted when they are based on outside text. AI should be used as a tool for convenience, but don’t let it catch you off guard.
To stay safe, remember to remain skeptical, don’t reply out of urgency, and try to verify if the threat is real. If the summary contains a worrying alert, open up the original email and give it a read. Check for any potential strange grammar or unusual senders. Make sure you never share passwords with anyone on the phone. If you need help, you can reach Google using the Help menu inside Gmail itself. You could also turn off email summarization until the protective measures have improved. If you do come across any suspicious emails, you can let Google know by selecting “Report phishing”. Every little bit helps make your online experience that much safer.
Read More: You Can (and Should) Delete Yourself From The Internet. Here’s How.