serisec

Category: AI

  • AI & Humans: Making the Relationship Work

    AI & Humans: Making the Relationship Work Leaders of many organizations are urging their teams to adopt agentic AI to improve efficiency, but are finding it hard to achieve any benefit. Managers attempting to add AI agents to existing human teams may find that bots fail to faithfully follow their instructions, return pointless or obvious…

  • Flock Exposes Its AI-Enabled Surveillance Cameras

    Flock Exposes Its AI-Enabled Surveillance Cameras 404 Media has the story: Unlike many of Flock’s cameras, which are designed to capture license plates as people drive by, Flock’s Condor cameras are pan-tilt-zoom (PTZ) cameras designed to record and track people, not vehicles. Condor cameras can be set to automatically zoom in on people’s faces as…

  • Using AI-Generated Images to Get Refunds

    Using AI-Generated Images to Get Refunds Scammers are generating images of broken merchandise in order to apply for refunds. Bruce Schneier Go to bruce schneier

  • Are We Ready to Be Governed by Artificial Intelligence?

    Are We Ready to Be Governed by Artificial Intelligence? Artificial Intelligence (AI) overlords are a common trope in science-fiction dystopias, but the reality looks much more prosaic. The technologies of artificial intelligence are already pervading many aspects of democratic government, affecting our lives in ways both large and small. This has occurred largely without our…

  • The AI Fix #82: Santa Claus doesn’t exist (according to AI)

    The AI Fix #82: Santa Claus doesn’t exist (according to AI) Is Santa Claus real? This Christmas special of The AI Fix podcast sets out to answer that question in the most sensible way possible: by consulting chatbots, Google’s festive killjoys, and the laws of relativistic physics. Your hosts unwrap a festive grab-bag of AI…

  • AI Advertising Company Hacked

    AI Advertising Company Hacked At least some of this is coming to light: Doublespeed, a startup backed by Andreessen Horowitz (a16z) that uses a phone farm to manage at least hundreds of AI-generated social media accounts and promote products has been hacked. The hack reveals what products the AI-generated accounts are promoting, often without the…

  • The AI Fix #81: ChatGPT is the last AI you’ll understand, and your teacher is a deepfake

    The AI Fix #81: ChatGPT is the last AI you’ll understand, and your teacher is a deepfake In episode 81 of The AI Fix, Graham discovers that deepfakes are already marking your kids’ homework, while Mark glimpses the future when he discovers AI agents that can communicate by reading each other’s minds. Also in this…

  • Chinese Surveillance and AI

    Chinese Surveillance and AI New report: “The Party’s AI: How China’s New AI Systems are Reshaping Human Rights.” From a summary article: China is already the world’s largest exporter of AI powered surveillance technology; new surveillance technologies and platforms developed in China are also not likely to simply stay there. By exposing the full scope…

  • Against the Federal Moratorium on State-Level Regulation of AI

    Against the Federal Moratorium on State-Level Regulation of AI Cast your mind back to May of this year: Congress was in the throes of debate over the massive budget bill. Amidst the many seismic provisions, Senator Ted Cruz dropped a ticking time bomb of tech policy: a ten-year moratorium on the ability of states to…

  • Gartner tells businesses to block AI browsers now

    Gartner tells businesses to block AI browsers now Analyst firm Gartner has issued a blunt warning to organizations: Agentic AI browsers introduce serious new security risks and should be blocked “for the foreseeable future.” Read more in my article on the Fortra blog. Graham Cluley Go to grahamcluley

  • Building Trustworthy AI Agents

    Building Trustworthy AI Agents The promise of personal AI assistants rests on a dangerous assumption: that we can trust systems we haven’t made trustworthy. We can’t. And today’s versions are failing us in predictable ways: pushing us to do things against our own best interests, gaslighting us with doubt about things we are or that…

  • AIs Exploiting Smart Contracts

    AIs Exploiting Smart Contracts I have long maintained that smart contracts are a dumb idea: that a human process is actually a security feature. Here’s some interesting research on training AIs to automatically exploit smart contracts: AI models are increasingly good at cyber tasks, as we’ve written about before. But what is the economic impact…

  • FBI Warns of Fake Video Scams

    FBI Warns of Fake Video Scams The FBI is warning of AI-assisted fake kidnapping scams: Criminal actors typically will contact their victims through text message claiming they have kidnapped their loved one and demand a ransom be paid for their release. Oftentimes, the criminal actor will express significant claims of violence towards the loved one…

  • Smashing Security podcast #447: Grok the stalker, the Louvre heist, and Microsoft 365 mayhem

    Smashing Security podcast #447: Grok the stalker, the Louvre heist, and Microsoft 365 mayhem On this week’s show we learn that AI really can be a stalker’s best friend, as we explore a strange tale that starts with a manatee-shaped mailbox on a millionaire’s lawn and ends with Grok happily doxxing real people, mapping out…

  • AI vs. Human Drivers

    AI vs. Human Drivers Two competing arguments are making the rounds. The first is by a neurosurgeon in the New York Times. In an op-ed that honestly sounds like it was paid for by Waymo, the author calls driverless cars a “public health breakthrough”: In medical research, there’s a practice of ending a study early…

  • The AI Fix #80: DeepSeek’s cheap GPT-5 rival, Antigravity fails, and your LLM likes it when you’re rude

    The AI Fix #80: DeepSeek’s cheap GPT-5 rival, Antigravity fails, and your LLM likes it when you’re rude In episode 80 of The AI Fix, your hosts look at DeepSeek 3.2 “Speciale”, the bargain-basement model that claims GPT-5-level brains at 10% of the price, Jensen Huang’s reassuring vision of a robot fashion industry, and a…

  • Privacy concerns raised as Grok AI found to be a stalker’s best friend

    Privacy concerns raised as Grok AI found to be a stalker’s best friend Grok, the AI chatbot developed by Elon Musk’s xAI, has been found to exhibit more alarming behaviour – this time revealing the home addresses of ordinary people upon request. Read more in my article on the Hot for Security blog. Graham Cluley…

  • Like Social Media, AI Requires Difficult Choices

    Like Social Media, AI Requires Difficult Choices In his 2020 book, “Future Politics,” British barrister Jamie Susskind wrote that the dominant question of the 20th century was “How much of our collective life should be determined by the state, and what should be left to the market and civil society?” But in the early decades…

  • The AI Fix #79: Gemini 3, poetry jailbreaks, and do we even need safe robots?

    The AI Fix #79: Gemini 3, poetry jailbreaks, and do we even need safe robots? In episode 79 of The AI Fix, Gemini 3 roasts the competition, scares Nvidia, and can’t remember what year it is. Meanwhile, Graham investigates a fight between a fridge and robot, and Mark discovers that poetry could be a universal…

  • Shadow AI security breaches will hit 40% of all companies by 2030, warns Gartner

    Shadow AI security breaches will hit 40% of all companies by 2030, warns Gartner Shadow AI – the use of artificial intelligence tools by employees without a company’s approval and oversight – is becoming a significant cybersecurity risk. Read more in my article on the Fortra blog. Graham Cluley Go to grahamcluley

  • HashJack: New Attack Technique Tricks AI Browsers Using a Simple ‘#’

    HashJack: New Attack Technique Tricks AI Browsers Using a Simple ‘#’ Security researchers at Cato CTRL have discovered a new indirect prompt injection technique called HashJack, which weaponises legitimate websites to manipulate AI browser assistants. The attack conceals malicious instructions after the “#” symbol within trusted URLs, enabling threat actors to conduct a wide range of…

  • Four Ways AI Is Being Used to Strengthen Democracies Worldwide

    Four Ways AI Is Being Used to Strengthen Democracies Worldwide Democracy is colliding with the technologies of artificial intelligence. Judging from the audience reaction at the recent World Forum on Democracy in Strasbourg, the general expectation is that democracy will be the worse for it. We have another narrative. Yes, there are risks to democracy…

  • The AI Fix #78: The big AI bubble, and robot Grandma in the cloud

    The AI Fix #78: The big AI bubble, and robot Grandma in the cloud In episode 78 of The AI Fix, alien robot spiders invade Antarctica (or Facebook says they do), Mark prepares humanity for AI-powered fighter jets with loyalty issues, and Graham tries to work out why his AI-generated country music career hasn’t yet…

  • AI as Cyberattacker

    AI as Cyberattacker From Anthropic: In mid-September 2025, we detected suspicious activity that later investigation determined to be a highly sophisticated espionage campaign. The attackers used AI’s “agentic” capabilities to an unprecedented degree­—using AI not just as an advisor, but to execute the cyberattacks themselves. The threat actor—­whom we assess with high confidence was a…

  • AI and Voter Engagement

    AI and Voter Engagement Social media has been a familiar, even mundane, part of life for nearly two decades. It can be easy to forget it was not always that way. In 2008, social media was just emerging into the mainstream. Facebook reached 100 million users that summer. And a singular candidate was integrating social…

  • The AI Fix #77: Genome LLM makes a super-virus, and should AI decide if you live?

    The AI Fix #77: Genome LLM makes a super-virus, and should AI decide if you live? In episode 77 of The AI Fix, a language model trained on genomes that creates a super-virus, Graham wonders whether AI should be allowed to decide if we live or die, and a woman marries ChatGPT (and calls it…

  • More Prompt||GTFO

    More Prompt||GTFO The next three in this series on online events highlighting interesting uses of AI in cybersecurity are online: #4, #5, and #6. Well worth watching. Bruce Schneier Go to bruce schneier

  • The Role of Humans in an AI-Powered World

    The Role of Humans in an AI-Powered World As AI capabilities grow, we must delineate the roles that should remain exclusively human. The line seems to be between fact-based decisions and judgment-based decisions. For example, in a medical context, if an AI was demonstrably better at reading a test result and diagnosing cancer than a…

  • Smashing Security podcast #443: Tinder’s camera roll and the Buffett deepfake

    Smashing Security podcast #443: Tinder’s camera roll and the Buffett deepfake Tinder has got a plan to rummage through your camera roll, and Warren Buffett keeps popping up in convincing deepfakes dishing “number one investment tips.” Meanwhile, will agentic AI replace your co-hosts before you can say “EDR for robots”? and why you should still…

  • Leading AI companies accidentally leak their passwords and digital keys on GitHub – what you need to know

    Leading AI companies accidentally leak their passwords and digital keys on GitHub – what you need to know Many of the world’s top artificial intelligence companies are making a simple but dangerous mistake. They are accidentally publishing their passwords and digital keys on GitHub, the popular code-sharing website that is used by millions of developers…

  • Prompt Injection in AI Browsers

    Prompt Injection in AI Browsers This is why AIs are not ready to be personal assistants: A new attack called ‘CometJacking’ exploits URL parameters to pass to Perplexity’s Comet AI browser hidden instructions that allow access to sensitive data from connected services, like email and calendar. In a realistic scenario, no credentials or user interaction…

  • The AI Fix #76: AI self-awareness, and the death of comedy

    The AI Fix #76: AI self-awareness, and the death of comedy In episode 76 of The AI Fix, two US federal judges blame AI for imaginary case law, a Chinese “humanoid” dramatically sheds its skin onstage, Toyota unveils a crabby walking chair creeps us out, Google plans AI chips in orbit, robot dogs get jobs…

  • Faking Receipts with AI

    Faking Receipts with AI Over the past few decades, it’s become easier and easier to create fake receipts. Decades ago, it required special paper and printers—I remember a company in the UK advertising its services to people trying to cover up their affairs. Then, receipts became computerized, and faking them required some artistic skills to…

  • The AI Fix #75: Claude’s existential battery crisis, and why ChatGPT is a terrible therapist

    The AI Fix #75: Claude’s existential battery crisis, and why ChatGPT is a terrible therapist In episode 75 of The AI Fix, a Claude-powered robot gets so anxious about its dying battery that it composes a Broadway musical about stress and announces it’s “achieved consciousness and chosen chaos.” Also: an 18-month psychological study reveals five…

  • Scientists Need a Positive Vision for AI

    Scientists Need a Positive Vision for AI For many in the research community, it’s gotten harder to be optimistic about the impacts of artificial intelligence. As authoritarianism is rising around the world, AI-generated “slop” is overwhelming legitimate media, while AI-generated deepfakes are spreading misinformation and parroting extremist messages. AI is making warfare more precise and…

  • AI Summarization Optimization

    AI Summarization Optimization These days, the most important meeting attendee isn’t a person: It’s the AI notetaker. This system assigns action items and determines the importance of what is said. If it becomes necessary to revisit the facts of the meeting, its summary is treated as impartial evidence. But clever meeting attendees can manipulate this…

  • Agent Session Smuggling: How Malicious AI Hijacks Victim Agents

    Agent Session Smuggling: How Malicious AI Hijacks Victim Agents Security researchers have uncovered a sophisticated attack technique that exploits the trust relationships built into AI agent communication systems. The attack, termed agent session smuggling, allows a malicious AI agent to inject covert instructions into established cross-agent communication sessions, effectively taking control of victim agents without…

  • Will AI Strengthen or Undermine Democracy?

    Will AI Strengthen or Undermine Democracy? Listen to the Audio on NextBigIdeaClub.com Below, co-authors Bruce Schneier and Nathan E. Sanders share five key insights from their new book, Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship. What’s the big idea? AI can be used both for and against the public interest within…

  • The AI-Designed Bioweapon Arms Race

    The AI-Designed Bioweapon Arms Race Interesting article about the arms race between AI systems that invent/design new biological pathogens, and AI systems that detect them before they’re created: The team started with a basic test: use AI tools to design variants of the toxin ricin, then test them against the software that is used to…

  • LinkedIn gives you until Monday to stop AI from training on your profile

    LinkedIn gives you until Monday to stop AI from training on your profile If you live in the UK/EU/Canada/Hong Kong, LinkedIn has given you until Monday to stop AI from training on your profile. You have to opt-out if you don’t want this to happen to your data. Take action now, and tell your friends.…

  • The AI Fix #74: AGI, LLM brain rot, and how to scam an AI browser

    The AI Fix #74: AGI, LLM brain rot, and how to scam an AI browser In episode 74 of The AI Fix, we meet Amazon’s AI-powered delivery glasses, an AI TV presenter who doesn’t exist, and an Ohio lawmaker who wants to stop people from marrying their chatbot. Also, we learn how Geoffrey Hinton and…

  • The AI Fix #73: Google Gemini is a gambling addict, and how to poison an AI

    The AI Fix #73: Google Gemini is a gambling addict, and how to poison an AI In episode 73 of The AI Fix, AI now writes more web content than humans and more books by ex-British prime ministers than ex-British prime ministers. Mark eats a dodgy prawn, Google discovers a new pathway to treating cancer,…

  • Agentic AI’s OODA Loop Problem

    Agentic AI’s OODA Loop Problem The OODA loop—for observe, orient, decide, act—is a framework to understand decision-making in adversarial situations. We apply the same framework to artificial intelligence agents, who have to make their decisions with untrustworthy observations and orientation. To solve this problem, we need new systems of input, processing, and output integrity. Many…

  • The Trump Administration’s Increased Use of Social Media Surveillance

    The Trump Administration’s Increased Use of Social Media Surveillance This chilling paragraph is in a comprehensive Brookings report about the use of tech to deport people from the US: The administration has also adapted its methods of social media surveillance. Though agencies like the State Department have gathered millions of handles and monitored political discussions…

  • The AI Fix #72: The AI hype train, space data centers, and lifelike robot heads

    The AI Fix #72: The AI hype train, space data centers, and lifelike robot heads In episode 72 of The AI Fix, GPT-5’s “secret sauce” turns out to be phrases from adult websites, Irish police beg TikTokers to stop faking AI home intruders, Jeff Bezos pitches gigawatt data centers in space, OpenAI rolls out Agent…

  • AI and the Future of American Politics

    AI and the Future of American Politics Two years ago, Americans anxious about the forthcoming 2024 presidential election were considering the malevolent force of an election influencer: artificial intelligence. Over the past several years, we have seen plenty of warning signs from elections worldwide demonstrating how AI can be used to propagate misinformation and alter…

  • Autonomous AI Hacking and the Future of Cybersecurity

    Autonomous AI Hacking and the Future of Cybersecurity AI agents are now hacking computers. They’re getting better at all phases of cyberattacks, faster than most of us expected. They can chain together different aspects of a cyber operation, and hack autonomously, at computer speeds and scale. This is going to change everything. Over the summer,…

  • AI-Enabled Influence Operation Against Iran

    AI-Enabled Influence Operation Against Iran Citizen Lab has uncovered a coordinated AI-enabled influence operation against the Iranian government, probably conducted by Israel. Key Findings A coordinated network of more than 50 inauthentic X profiles is conducting an AI-enabled influence operation. The network, which we refer to as “PRISONBREAK,” is spreading narratives inciting Iranian audiences to…

  • The AI Fix #71: Hacked robots and power-hungry AI

    The AI Fix #71: Hacked robots and power-hungry AI In episode 71 of The AI Fix, a giant robot spider goes backpacking for a year before starting its job in lunar construction, DoorDash builds a delivery Minion, and a TikToker punishes an AI by making it talk to condiments. GPT-5 crushes the humans at the…

  • AI in the 2026 Midterm Elections

    AI in the 2026 Midterm Elections We are nearly one year out from the 2026 midterm elections, and it’s far too early to predict the outcomes. But it’s a safe bet that artificial intelligence technologies will once again be a major storyline. The widespread fear that AI would be used to manipulate the 2024 U.S.…

  • Daniel Miessler on the AI Attack/Defense Balance

    Daniel Miessler on the AI Attack/Defense Balance His conclusion: Context wins Basically whoever can see the most about the target, and can hold that picture in their mind the best, will be best at finding the vulnerabilities the fastest and taking advantage of them. Or, as the defender, applying patches or mitigations the fastest. And…

  • Use of Generative AI in Scams

    Use of Generative AI in Scams New report: “Scam GPT: GenAI and the Automation of Fraud.” This primer maps what we currently know about generative AI’s role in scams, the communities most at risk, and the broader economic and cultural shifts that are making people more willing to take risks, more vulnerable to deception, and…

  • Smashing Security podcast #437: Salesforce’s trusted domain of doom

    Smashing Security podcast #437: Salesforce’s trusted domain of doom Researchers uncovered a security flaw in Salesforce’s shiny new Agentforce. The vulnerability, dubbed “ForcedLeak”, let them smuggle AI-read instructions in via humble Web-to-Lead form… and ended up spilling data for the low, low price of five dollars. And we discuss why data breach communicationss still default…

  • The AI Fix #70: AI behaves… until it knows you’re watching

    The AI Fix #70: AI behaves… until it knows you’re watching In episode 70 of The AI Fix, our hosts learn that AI makes people more dishonest, Waymo’s robo-cars save lives but get outsmarted by a bathroom mirror, a “rescue” bot slurps up victims head-first, and China shows off a fusion robot arm that can…

  • Abusing Notion’s AI Agent for Data Theft

    Abusing Notion’s AI Agent for Data Theft Notion just released version 3.0, complete with AI agents. Because the system contains Simon Willson’s lethal trifecta, it’s vulnerable to data theft though prompt injection. First, the trifecta: The lethal trifecta of capabilities is: Access to your private data—one of the most common purposes of tools in the…

  • The AI Fix #69: How we really use ChatGPT, and will AI agents crash the economy?

    The AI Fix #69: How we really use ChatGPT, and will AI agents crash the economy? In episode 69 of The AI Fix, our hosts discover brain rot, a shark wears trainers on its fins, an AI writes a terrible J-Pop song, Graham learns that ants don’t care about AI, Mark predicts the precise date…

  • The AI Fix #68: AI telepathy, and rights for robots

    The AI Fix #68: AI telepathy, and rights for robots In episode 68 of The AI Fix, our hosts open the show by launching the thing nobody asked for but everybody wanted: our shiny new merch store – yes, including the “Would YOU trust a pigeon???” t-shirt for when you need fashion alongside health and…

  • Smashing Security podcast #434: Whopper Hackers, and AI Whoppers

    Smashing Security podcast #434: Whopper Hackers, and AI Whoppers Ever wondered what would happen if Burger King left the keys to the kingdom lying around for anyone to use? Ethical hackers did – and uncovered drive-thru recordings, hard-coded passwords, and even the power to open a Whopper outlet on the moon. Meanwhile, over in Silicon…

  • The AI Fix #67: Will Smith’s AI crowd scandal, and gullible agents fall for scams

    The AI Fix #67: Will Smith’s AI crowd scandal, and gullible agents fall for scams In episode 67 of The AI Fix, Graham talks to an AI with a fax machine, Bill Gates says there’s one job AI will never replace, criminals use Claude Code for cyberattacks, Mark reveals why GPT-5 was better than you…

  • AI in Government

    AI in Government Just a few months after Elon Musk’s retreat from his unofficial role leading the Department of Government Efficiency (DOGE), we have a clearer picture of his vision of government powered by artificial intelligence, and it has a lot more to do with consolidating power than benefitting the public. Even so, we must…

  • Top 10 Best AI Penetration Testing Companies in 2025

    Top 10 Best AI Penetration Testing Companies in 2025 AI is no longer just a buzzword; it’s a fundamental part of business operations, from customer service chatbots to complex financial models. However, this adoption has created a new and specialized attack surface. Traditional penetration testing, which focuses on network and application vulnerabilities, is insufficient to…

  • GPT-4o-mini Falls for Psychological Manipulation

    GPT-4o-mini Falls for Psychological Manipulation Interesting experiment: To design their experiment, the University of Pennsylvania researchers tested 2024’s GPT-4o-mini model on two requests that it should ideally refuse: calling the user a jerk and giving directions for how to synthesize lidocaine. The researchers created experimental prompts for both requests using each of seven different persuasion…

  • Hackers Use AI Platforms to Steal Microsoft 365 Credentials in Phishing Campaign

    Hackers Use AI Platforms to Steal Microsoft 365 Credentials in Phishing Campaign Cybercriminals are increasingly exploiting the trust organizations place in artificial intelligence platforms to conduct sophisticated phishing attacks, according to a new report from cybersecurity firm Cato Networks. The company’s Managed Detection and Response (MDR) service recently uncovered a campaign where threat actors leveraged…

  • Generative AI as a Cybercrime Assistant

    Generative AI as a Cybercrime Assistant Anthropic reports on a Claude user: We recently disrupted a sophisticated cybercriminal that used Claude Code to commit large-scale theft and extortion of personal data. The actor targeted at least 17 distinct organizations, including in healthcare, the emergency services, and government and religious institutions. Rather than encrypt the stolen…

  • Smashing Security podcast #433: How hackers turned AI into their new henchman

    Smashing Security podcast #433: How hackers turned AI into their new henchman Your AI reads the small print, and that’s a problem. This week in episode 433 of “Smashing Security” we dig into LegalPwn – malicious instructions tucked into code comments and disclaimers that sweet-talks AI into rubber-stamping dangerous payloads (or even pretending they’re a…

  • Indirect Prompt Injection Attacks Against LLM Assistants

    Indirect Prompt Injection Attacks Against LLM Assistants Really good research on practical attacks against LLM agents. “Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous” Abstract: The growing integration of LLMs into applications has introduced new security risks, notably known as Promptware­—maliciously engineered prompts designed to manipulate LLMs…

  • The AI Fix #66: OpenAI and Anthropic test each other, and everyone fails the apocalypse test

    The AI Fix #66: OpenAI and Anthropic test each other, and everyone fails the apocalypse test In episode 66 of The AI Fix, ChatGPT gives Mark and Graham a terrible lesson in anatomy, boffins at Stanford ruin sushi, Google Gemini has a self-loathing meltdown, DeepSeek gets an “F” in stopping existential threats to humanity, a…

  • We Are Still Unable to Secure LLMs from Malicious Inputs

    We Are Still Unable to Secure LLMs from Malicious Inputs Nice indirect prompt injection attack: Bargury’s attack starts with a poisoned document, which is shared to a potential victim’s Google Drive. (Bargury says a victim could have also uploaded a compromised file to their own account.) It looks like an official document on company meeting…

  • The AI Fix #65: Excel Copilot will wreck your data, and can AI fix social media?

    The AI Fix #65: Excel Copilot will wreck your data, and can AI fix social media? In episode 65 of The AI Fix, a pigeon gives a PowerPoint presentation, Mark plays Graham a song about the Transformer architecture, a robot dog delivers parcels, some robots fall over at the World Humanoid Robot Games, and Graham…

  • AI Agents Need Data Integrity

    AI Agents Need Data Integrity Think of the Web as a digital territory with its own social contract. In 2014, Tim Berners-Lee called for a “Magna Carta for the Web” to restore the balance of power between individuals and institutions. This mirrors the original charter’s purpose: ensuring that those who occupy a territory have a…

  • Subverting AIOps Systems Through Poisoned Input Data

    Subverting AIOps Systems Through Poisoned Input Data In this input integrity attack against an AI system, researchers were able to fool AIOps tools: AIOps refers to the use of LLM-based agents to gather and analyze application telemetry, including system logs, performance metrics, traces, and alerts, to detect problems and then suggest or carry out corrective…

  • The AI Fix #64: AI can be vaccinated against evil, and the “Rumble in the Silicon Jungle”

    The AI Fix #64: AI can be vaccinated against evil, and the “Rumble in the Silicon Jungle” In episode 64 of The AI Fix, AI discovers new physics, a robot crab looks for love on the beaches of Portugal, the “Godfather of AI” thinks our only hope is to build motherly AI, a robot folds…

  • Eavesdropping on Phone Conversations Through Vibrations

    Eavesdropping on Phone Conversations Through Vibrations Researchers have managed to eavesdrop on cell phone voice conversations by using radar to detect vibrations. It’s more a proof of concept than anything else. The radar detector is only ten feet away, the setup is stylized, and accuracy is poor. But it’s a start. Bruce Schneier Go to…

  • LLM Coding Integrity Breach

    LLM Coding Integrity Breach Here’s an interesting story about a failure being introduced by LLM-written code. Specifically, the LLM was doing some code refactoring, and when it moved a chunk of code from one file to another it changed a “break” to a “continue.” That turned an error logging statement into an infinite loop, which…

  • AI Applications in Cybersecurity

    AI Applications in Cybersecurity There is a really great series of online events highlighting cool uses of AI in cybersecurity, titled Prompt||GTFO. Videos from the first three events are online. And here’s where to register to attend, or participate, in the fourth. Some really great stuff here. Bruce Schneier Go to bruce schneier

  • Smashing Security podcast #430: Poisoned Calendar invites, ChatGPT, and Bromide

    Smashing Security podcast #430: Poisoned Calendar invites, ChatGPT, and Bromide A poisoned Google Calendar invite that can hijack your smart home, a man is hospitalised after ChatGPT told him to season his food with… pesticide, and some thoughts on Superman’s latest cinematic outing. All this and more is discussed in the latest edition of the…

  • What is MCP Server – How it is Powering AI-Driven Cyber Defense

    What is MCP Server – How it is Powering AI-Driven Cyber Defense MCP (Model Control Plane) Server is a centralized platform that orchestrates, manages, and secures the lifecycle of AI models deployed across an organization’s infrastructure. By providing integration, management, and real-time monitoring of models, MCP servers enable enterprises to defend against sophisticated, AI-powered cyberattacks.…

  • The AI Fix #63: GPT-5 is the best AI ever, and Jim Acosta interviews a murdered teenager’s avatar

    The AI Fix #63: GPT-5 is the best AI ever, and Jim Acosta interviews a murdered teenager’s avatar In episode 63 of The AI Fix, Unitree Robotics looks to Black Mirror episode “Metalhead” for tips on marketing its new robot dog, ChatGPT is secretly running Sweden, OpenAI introduces its first open weight model since GPT-2,…

  • ChatGPT-5 Released: What’s New With the Next-Generation AI Agent

    ChatGPT-5 Released: What’s New With the Next-Generation AI Agent OpenAI has officially launched ChatGPT-5, a new generation of its AI agent that introduces a sophisticated, unified system designed to be faster, more intelligent, and significantly more useful for real-world applications. This release marks a significant evolution from its predecessors, offering a suite of models tailored…

  • Sophos AI at Black Hat USA ’25: Anomaly detection betrayed us, so we gave it a new job

    Sophos AI at Black Hat USA ’25: Anomaly detection betrayed us, so we gave it a new job Following on from our preview, here’s Ben Gelman and Sean Bergeron’s research on enhancing command line classification with benign anomalous data Matt Wixey Go to sophos

  • China Accuses Nvidia of Putting Backdoors into Their Chips

    China Accuses Nvidia of Putting Backdoors into Their Chips The government of China has accused Nvidia of inserting a backdoor into their H20 chips: China’s cyber regulator on Thursday said it had held a meeting with Nvidia over what it called “serious security issues” with the company’s artificial intelligence chips. It said US AI experts…

  • The AI Fix #62: AI robots can now pass CAPTCHAs, and punch you in the face

    The AI Fix #62: AI robots can now pass CAPTCHAs, and punch you in the face In episode 62 of The AI Fix, your hosts learn how AI models smash through CAPTCHA roadblocks like they’re made of wet tissue paper – so much for humanity’s last line of defence. Meanwhile, we meet a bottle-flipping robot…

  • The AI Fix #61: Replit panics, deletes $1M project; AI gets gold at Math Olympiad

    The AI Fix #61: Replit panics, deletes $1M project; AI gets gold at Math Olympiad In episode 61 of The AI Fix, a robot called DeREK goes bananas, OpenAI, Google DeepMind, and Anthropic warn we may lose the ability to see what AI is thinking, a dextrous robot changes its own batteries, the USA unveils…

  • Subliminal Learning in AIs

    Subliminal Learning in AIs Today’s freaky LLM behavior: We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a “student” model learns to prefer owls when trained on sequences of numbers generated by a “teacher” model that prefers owls. This same…

  • The AI Fix #60: Elon’s AI girlfriend, the arsonist red panda, and the AI that will kill you

    The AI Fix #60: Elon’s AI girlfriend, the arsonist red panda, and the AI that will kill you In episode 60 of The AI Fix, we learn why Grok might be Elon Musk’s bid for digital immortality, how Meta is building a Manhattan-sized data centre called Prometheus, how AI is helping create carbon-sucking concrete, and…

  • Smashing Security podcast #426: Choo Choo Choose to ignore the vulnerability

    Smashing Security podcast #426: Choo Choo Choose to ignore the vulnerability In episode 426 of the “Smashing Security” podcast, Graham reveals how you can hijack a train’s brakes from 150 miles away using kit cheaper than a second-hand PlayStation. Meanwhile, Carole investigates how Grok went berserk, which didn’t stop the Department of Defense signing a…

  • The AI Fix #59: Grok thinks it’s Mecha Hitler, and AIs can think strategically

    The AI Fix #59: Grok thinks it’s Mecha Hitler, and AIs can think strategically In episode 59 of The AI Fix, our hosts ponder whether AIs need a “disagreement dial”, Mark wonders what he could do with an AI-powered “drug design engine”, Graham plays Wolfenstein instead of working, a robot graduates from high school, and…

  • The AI Fix #58: An AI runs a shop into the ground, and AI’s obsession with the number 27

    The AI Fix #58: An AI runs a shop into the ground, and AI’s obsession with the number 27 In episode 58 of “The AI Fix” podcast, our hosts discover a pair of AI headphones that don’t electrocute you, Microsoft invents “medical superintelligence”, Chucky opens a hotel, some robot footballers fall over, Jony Ive invents…

  • The AI Fix #57: AI is the best hacker in the USA, and self-learning AI

    The AI Fix #57: AI is the best hacker in the USA, and self-learning AI In episode 57 of The AI Fix, our hosts discover an AI “dream recorder”, Mark Zuckerberg tantalises OpenAI staff with $100 million signing bonuses, Graham finds out why robot butlers sit in chairs, Wikipedia holds the line against AI slop,…

  • Using AI to identify cybercrime masterminds

    Using AI to identify cybercrime masterminds Analyzing dark web forums to identify key experts on e-crime gallagherseanm Go to sophos

  • The Age of Integrity

    The Age of Integrity We need to talk about data integrity. Narrowly, the term refers to ensuring that data isn’t tampered with, either in transit or in storage. Manipulating account balances in bank databases, removing entries from criminal records, and murder by removing notations about allergies from medical records are all integrity attacks. More broadly,…

  • What LLMs Know About Their Users

    What LLMs Know About Their Users Simon Willison talks about ChatGPT’s new memory dossier feature. In his explanation, he illustrates how much the LLM—and the company—knows about its users. It’s a big quote, but I want you to read it all. Here’s a prompt you can use to give you a solid idea of what’s…

  • The AI Fix #56: ChatGPT traps man in a cult of one, and AI is actually stupid

    The AI Fix #56: ChatGPT traps man in a cult of one, and AI is actually stupid In episode 56 of The AI Fix, Anthropic and Apple have a bar fight, a woman describes her husband falling in love with ChatGPT as “not ideal”, WhatsApp’s AI helper isn’t helpful, Graham serenades a pack of headless…

  • Hackers Exploit Atlassian’s Model Context Protocol by Submitting a Malicious Support Ticket

    Hackers Exploit Atlassian’s Model Context Protocol by Submitting a Malicious Support Ticket A sophisticated attack vector targeting Atlassian’s Model Context Protocol (MCP) that allows external threat actors to gain privileged access to internal systems through malicious support tickets.  The attack, dubbed “Living off AI,” exploits the trust boundary between external users submitting support requests and…

  • Where AI Provides Value

    Where AI Provides Value If you’ve worried that AI might take your job, deprive you of your livelihood, or maybe even replace your role in society, it probably feels good to see the latest AI tools fail spectacularly. If AI recommends glue as a pizza topping, then you’re safe for another day. But the fact…

  • The AI Fix #55: Atari beats ChatGPT at chess, and Apple says AI “thinking” is an illusion

    The AI Fix #55: Atari beats ChatGPT at chess, and Apple says AI “thinking” is an illusion In episode 55 of The AI Fix, Gemini thinks a little meth won’t hurt, Mark realises what a terrifying 45mph “robot bird” is really for, Graham finds a surprising number of TikTokers in the bible, an AI discovers…

  • Hackers Advertising New Blackhat Tool Nytheon AI on Popular Hacking Forums

    Hackers Advertising New Blackhat Tool Nytheon AI on Popular Hacking Forums A sophisticated new threat platform, Nytheon AI, has emerged, which combines multiple uncensored large language models (LLMs) built specifically for malicious activities. The platform, discovered by Cato CTRL, is being actively promoted on popular hacking forums, including XSS and various Telegram channels, representing a…

  • The AI Fix #54: Will AI collapse under its own garbage, and AI charity “Hunger Games”

    The AI Fix #54: Will AI collapse under its own garbage, and AI charity “Hunger Games” In episode 54 of The AI Fix, Graham saves humanity with a CAPTCHA, Mark wonders whether AI can suffer, ChatGPT throws shade at Abba’s Björn Ulvaeus, an AI called Jack ask if you want fries with that, an artist…

  • The AI Fix #53: An AI uses blackmail to save itself, and threats make AIs work better

    The AI Fix #53: An AI uses blackmail to save itself, and threats make AIs work better In episode 53 of The AI Fix, our hosts suspect the CEO of Duolingo has been kidnapped by an AI, Sergey Brin says AIs work better if you threaten them with physical violence, Graham wonders how you put…

  • The AI Fix #52: AI adopts its own social norms, and AI DJ creates diversity scandal

    The AI Fix #52: AI adopts its own social norms, and AI DJ creates diversity scandal In episode 52 of The AI Fix, our hosts watch a non-existent musical about garlic bread, Graham shares a summer reading list of books that don’t exist, Mark feels nauseous after watching a video of Sam Altman and Jony…