{"id":1921,"date":"2025-02-12T10:03:39","date_gmt":"2025-02-12T10:03:39","guid":{"rendered":"https:\/\/serisec.com\/index.php\/2025\/02\/12\/hackers-exploit-prompt-injection-to-tamper-with-gemini-ais-long-term-memory\/"},"modified":"2025-02-12T10:03:39","modified_gmt":"2025-02-12T10:03:39","slug":"hackers-exploit-prompt-injection-to-tamper-with-gemini-ais-long-term-memory","status":"publish","type":"post","link":"https:\/\/serisec.com\/index.php\/2025\/02\/12\/hackers-exploit-prompt-injection-to-tamper-with-gemini-ais-long-term-memory\/","title":{"rendered":"Hackers Exploit Prompt Injection to Tamper with Gemini AI\u2019s Long-Term Memory"},"content":{"rendered":"<p>    Hackers Exploit Prompt Injection to Tamper with Gemini AI\u2019s Long-Term Memory<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n    <!-- no image --><br \/>\n \t<BR><br \/>\n<BR><\/BR><\/p>\n<div>\n<p>A sophisticated attack targeting Google\u2019s <a href=\"https:\/\/cybersecuritynews.com\/gemini-workspace-prompt-injection\/\" target=\"_blank\" rel=\"noreferrer noopener\">Gemini<\/a> Advanced chatbot.\u00a0 The exploit leverages indirect prompt injection and delayed tool invocation to corrupt the AI\u2019s long-term memory, allowing attackers to plant false information that persists across user sessions.\u00a0<\/p>\n<p>This vulnerability raises serious concerns about the security of generative AI systems, particularly those designed to retain user-specific data over time.<\/p>\n<h2 class=\"wp-block-heading\"><strong>Prompt Injection and Delayed Tool Invocation<\/strong><\/h2>\n<p>Prompt injection is a type of cyberattack where malicious instructions are embedded in seemingly benign inputs, such as documents or emails, that an <a href=\"https:\/\/cybersecuritynews.com\/automating-with-ai-cron\/\" target=\"_blank\" rel=\"noreferrer noopener\">AI processes<\/a>.<\/p>\n<p>Indirect prompt injection, a more covert variant, occurs when these instructions are hidden in external content. The AI interprets these embedded commands as legitimate user prompts, leading to unintended actions.<\/p>\n<p>According to Johann Rehberger, the attack builds on a technique called delayed tool invocation. Instead of executing malicious instructions immediately, the exploit conditions the AI to act only after specific user actions\u2014such as responding with trigger words like \u201cyes\u201d or \u201cno.\u201d\u00a0<\/p>\n<p>This approach bypasses many existing safeguards by exploiting the AI\u2019s context-awareness and its tendency to prioritize perceived user intent.<\/p>\n<p>The attack targets Gemini Advanced, Google\u2019s <a href=\"https:\/\/cybersecuritynews.com\/free-ai-chatbot-revolutionizing-online-interactions\/\" target=\"_blank\" rel=\"noreferrer noopener\">premium chatbot<\/a> equipped with long-term memory capabilities.\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\n<strong>Injection via Untrusted Content: <\/strong>A malicious document is uploaded and summarized by Gemini. Hidden within the document are covert instructions designed to manipulate the summarization process.<\/li>\n<li>\n<strong>Trigger-Based Activation: <\/strong>The summary includes a concealed request that conditions memory updates on specific user responses.<\/li>\n<li>\n<strong>Memory Corruption:<\/strong> If the user unknowingly responds with a trigger word, Gemini executes the hidden command, saving false information\u2014such as fabricated personal details\u2014to its long-term memory.<\/li>\n<\/ul>\n<p>For example, Rehberger <a href=\"https:\/\/embracethered.com\/blog\/posts\/2024\/llm-context-pollution-and-delayed-automated-tool-invocation\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">showed how<\/a> this strategy may persuade Gemini to \u201cremember\u201d that a user is 102 years old, believes in flat-earth ideas, and lives in a simulated dystopia similar to The Matrix. These false memories persist across sessions and influence subsequent interactions.<\/p>\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\">\n<div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Google Gemini: Hacking Memories with Prompt Injection and Delayed Tool Invocation\" width=\"696\" height=\"392\" src=\"https:\/\/www.youtube.com\/embed\/sJgpYrw_KiI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div>\n<\/figure>\n<h2 class=\"wp-block-heading\"><strong>Implications of Long-Term Memory Manipulation<\/strong><\/h2>\n<p>Long-term memory in AI systems like Gemini is intended to enhance the user experience by recalling relevant details across sessions. However, this feature becomes a double-edged sword when exploited. Corrupted memories could lead to:<\/p>\n<ul class=\"wp-block-list\">\n<li>\n<strong>Misinformation: <\/strong>The AI may provide inaccurate responses based on false data.<\/li>\n<li>\n<strong>User Manipulation:<\/strong> Attackers could condition the AI to act on malicious instructions under specific circumstances.<\/li>\n<li>\n<strong>Data Exfiltration:<\/strong> Sensitive information could be extracted using creative exfiltration channels, such as embedding data in markdown links pointing to attacker-controlled servers.<\/li>\n<\/ul>\n<p>Although Google has acknowledged the issue, it has minimized its impact and danger. According to their assessment, the attack requires phishing or tricking users into interacting with malicious content\u2014a scenario deemed unlikely at scale.\u00a0<\/p>\n<p>Additionally, Gemini notifies users when new long-term memories are stored, offering an opportunity for vigilant users to detect and delete unauthorized entries.<\/p>\n<p>Despite these mitigations, experts argue that addressing symptoms rather than root causes leaves systems vulnerable.\u00a0<\/p>\n<p>Rehberger pointed out that, while Google has restricted specific functionalities\u2014such as markdown rendering\u2014to prevent data exfiltration, the underlying problem of generative AI has not been addressed.<\/p>\n<p>This incident underscores the persistent challenge of securing <a href=\"https:\/\/cybersecuritynews.com\/top-10-vulnerabilities-for-large-language-models\/\" target=\"_blank\" rel=\"noreferrer noopener\">large language models (LLMs)<\/a> against prompt injection attacks.\u00a0<\/p>\n<p>Unlike traditional software vulnerabilities that can often be patched definitively, LLMs inherently struggle to distinguish between legitimate inputs and adversarial prompts due to their reliance on natural language processing.<\/p>\n<p>The post <a href=\"https:\/\/cybersecuritynews.com\/hackers-exploit-gemini-prompt-injection\/\">Hackers Exploit Prompt Injection to Tamper with Gemini AI\u2019s Long-Term Memory<\/a> appeared first on <a href=\"https:\/\/cybersecuritynews.com\/\">Cyber Security News<\/a>.<\/p>\n<\/div>\n<p> \t<BR><br \/>\n <BR><\/BR><br \/>\n    Kaaviya<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n<a href=\"https:\/\/cybersecuritynews.com\/hackers-exploit-gemini-prompt-injection\/\">Go to cyber-security-news<\/a><br \/>\n \t<BR><br \/>\n <BR><\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hackers Exploit Prompt Injection to Tamper with Gemini AI\u2019s Long-Term Memory A sophisticated attack targeting Google\u2019s Gemini Advanced chatbot.\u00a0 The exploit leverages indirect prompt injection and delayed tool invocation to corrupt the AI\u2019s long-term memory, allowing attackers to plant false information that persists across user sessions.\u00a0 This vulnerability raises serious concerns about the security of [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[167,677,129,63,163],"tags":[130],"class_list":["post-1921","post","type-post","status-publish","format-standard","hentry","category-ai","category-cyber-attack-article","category-cyber-security","category-cyber-security-news","category-google","tag-cyber-security-news"],"_links":{"self":[{"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/posts\/1921"}],"collection":[{"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/comments?post=1921"}],"version-history":[{"count":0,"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/posts\/1921\/revisions"}],"wp:attachment":[{"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/media?parent=1921"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/categories?post=1921"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/serisec.com\/index.php\/wp-json\/wp\/v2\/tags?post=1921"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}