serisec

Category: AI

  • The AI Fix #77: Genome LLM makes a super-virus, and should AI decide if you live?

    The AI Fix #77: Genome LLM makes a super-virus, and should AI decide if you live? In episode 77 of The AI Fix, a language model trained on genomes that creates a super-virus, Graham wonders whether AI should be allowed to decide if we live or die, and a woman marries ChatGPT (and calls it…

  • More Prompt||GTFO

    More Prompt||GTFO The next three in this series on online events highlighting interesting uses of AI in cybersecurity are online: #4, #5, and #6. Well worth watching. Bruce Schneier Go to bruce schneier

  • The Role of Humans in an AI-Powered World

    The Role of Humans in an AI-Powered World As AI capabilities grow, we must delineate the roles that should remain exclusively human. The line seems to be between fact-based decisions and judgment-based decisions. For example, in a medical context, if an AI was demonstrably better at reading a test result and diagnosing cancer than a…

  • Smashing Security podcast #443: Tinder’s camera roll and the Buffett deepfake

    Smashing Security podcast #443: Tinder’s camera roll and the Buffett deepfake Tinder has got a plan to rummage through your camera roll, and Warren Buffett keeps popping up in convincing deepfakes dishing “number one investment tips.” Meanwhile, will agentic AI replace your co-hosts before you can say “EDR for robots”? and why you should still…

  • Leading AI companies accidentally leak their passwords and digital keys on GitHub – what you need to know

    Leading AI companies accidentally leak their passwords and digital keys on GitHub – what you need to know Many of the world’s top artificial intelligence companies are making a simple but dangerous mistake. They are accidentally publishing their passwords and digital keys on GitHub, the popular code-sharing website that is used by millions of developers…

  • Prompt Injection in AI Browsers

    Prompt Injection in AI Browsers This is why AIs are not ready to be personal assistants: A new attack called ‘CometJacking’ exploits URL parameters to pass to Perplexity’s Comet AI browser hidden instructions that allow access to sensitive data from connected services, like email and calendar. In a realistic scenario, no credentials or user interaction…

  • The AI Fix #76: AI self-awareness, and the death of comedy

    The AI Fix #76: AI self-awareness, and the death of comedy In episode 76 of The AI Fix, two US federal judges blame AI for imaginary case law, a Chinese “humanoid” dramatically sheds its skin onstage, Toyota unveils a crabby walking chair creeps us out, Google plans AI chips in orbit, robot dogs get jobs…

  • Faking Receipts with AI

    Faking Receipts with AI Over the past few decades, it’s become easier and easier to create fake receipts. Decades ago, it required special paper and printers—I remember a company in the UK advertising its services to people trying to cover up their affairs. Then, receipts became computerized, and faking them required some artistic skills to…

  • The AI Fix #75: Claude’s existential battery crisis, and why ChatGPT is a terrible therapist

    The AI Fix #75: Claude’s existential battery crisis, and why ChatGPT is a terrible therapist In episode 75 of The AI Fix, a Claude-powered robot gets so anxious about its dying battery that it composes a Broadway musical about stress and announces it’s “achieved consciousness and chosen chaos.” Also: an 18-month psychological study reveals five…

  • Scientists Need a Positive Vision for AI

    Scientists Need a Positive Vision for AI For many in the research community, it’s gotten harder to be optimistic about the impacts of artificial intelligence. As authoritarianism is rising around the world, AI-generated “slop” is overwhelming legitimate media, while AI-generated deepfakes are spreading misinformation and parroting extremist messages. AI is making warfare more precise and…

  • AI Summarization Optimization

    AI Summarization Optimization These days, the most important meeting attendee isn’t a person: It’s the AI notetaker. This system assigns action items and determines the importance of what is said. If it becomes necessary to revisit the facts of the meeting, its summary is treated as impartial evidence. But clever meeting attendees can manipulate this…

  • Agent Session Smuggling: How Malicious AI Hijacks Victim Agents

    Agent Session Smuggling: How Malicious AI Hijacks Victim Agents Security researchers have uncovered a sophisticated attack technique that exploits the trust relationships built into AI agent communication systems. The attack, termed agent session smuggling, allows a malicious AI agent to inject covert instructions into established cross-agent communication sessions, effectively taking control of victim agents without…

  • Will AI Strengthen or Undermine Democracy?

    Will AI Strengthen or Undermine Democracy? Listen to the Audio on NextBigIdeaClub.com Below, co-authors Bruce Schneier and Nathan E. Sanders share five key insights from their new book, Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship. What’s the big idea? AI can be used both for and against the public interest within…

  • The AI-Designed Bioweapon Arms Race

    The AI-Designed Bioweapon Arms Race Interesting article about the arms race between AI systems that invent/design new biological pathogens, and AI systems that detect them before they’re created: The team started with a basic test: use AI tools to design variants of the toxin ricin, then test them against the software that is used to…

  • LinkedIn gives you until Monday to stop AI from training on your profile

    LinkedIn gives you until Monday to stop AI from training on your profile If you live in the UK/EU/Canada/Hong Kong, LinkedIn has given you until Monday to stop AI from training on your profile. You have to opt-out if you don’t want this to happen to your data. Take action now, and tell your friends.…

  • The AI Fix #74: AGI, LLM brain rot, and how to scam an AI browser

    The AI Fix #74: AGI, LLM brain rot, and how to scam an AI browser In episode 74 of The AI Fix, we meet Amazon’s AI-powered delivery glasses, an AI TV presenter who doesn’t exist, and an Ohio lawmaker who wants to stop people from marrying their chatbot. Also, we learn how Geoffrey Hinton and…

  • The AI Fix #73: Google Gemini is a gambling addict, and how to poison an AI

    The AI Fix #73: Google Gemini is a gambling addict, and how to poison an AI In episode 73 of The AI Fix, AI now writes more web content than humans and more books by ex-British prime ministers than ex-British prime ministers. Mark eats a dodgy prawn, Google discovers a new pathway to treating cancer,…

  • Agentic AI’s OODA Loop Problem

    Agentic AI’s OODA Loop Problem The OODA loop—for observe, orient, decide, act—is a framework to understand decision-making in adversarial situations. We apply the same framework to artificial intelligence agents, who have to make their decisions with untrustworthy observations and orientation. To solve this problem, we need new systems of input, processing, and output integrity. Many…

  • The Trump Administration’s Increased Use of Social Media Surveillance

    The Trump Administration’s Increased Use of Social Media Surveillance This chilling paragraph is in a comprehensive Brookings report about the use of tech to deport people from the US: The administration has also adapted its methods of social media surveillance. Though agencies like the State Department have gathered millions of handles and monitored political discussions…

  • The AI Fix #72: The AI hype train, space data centers, and lifelike robot heads

    The AI Fix #72: The AI hype train, space data centers, and lifelike robot heads In episode 72 of The AI Fix, GPT-5’s “secret sauce” turns out to be phrases from adult websites, Irish police beg TikTokers to stop faking AI home intruders, Jeff Bezos pitches gigawatt data centers in space, OpenAI rolls out Agent…

  • AI and the Future of American Politics

    AI and the Future of American Politics Two years ago, Americans anxious about the forthcoming 2024 presidential election were considering the malevolent force of an election influencer: artificial intelligence. Over the past several years, we have seen plenty of warning signs from elections worldwide demonstrating how AI can be used to propagate misinformation and alter…

  • Autonomous AI Hacking and the Future of Cybersecurity

    Autonomous AI Hacking and the Future of Cybersecurity AI agents are now hacking computers. They’re getting better at all phases of cyberattacks, faster than most of us expected. They can chain together different aspects of a cyber operation, and hack autonomously, at computer speeds and scale. This is going to change everything. Over the summer,…

  • AI-Enabled Influence Operation Against Iran

    AI-Enabled Influence Operation Against Iran Citizen Lab has uncovered a coordinated AI-enabled influence operation against the Iranian government, probably conducted by Israel. Key Findings A coordinated network of more than 50 inauthentic X profiles is conducting an AI-enabled influence operation. The network, which we refer to as “PRISONBREAK,” is spreading narratives inciting Iranian audiences to…

  • The AI Fix #71: Hacked robots and power-hungry AI

    The AI Fix #71: Hacked robots and power-hungry AI In episode 71 of The AI Fix, a giant robot spider goes backpacking for a year before starting its job in lunar construction, DoorDash builds a delivery Minion, and a TikToker punishes an AI by making it talk to condiments. GPT-5 crushes the humans at the…

  • AI in the 2026 Midterm Elections

    AI in the 2026 Midterm Elections We are nearly one year out from the 2026 midterm elections, and it’s far too early to predict the outcomes. But it’s a safe bet that artificial intelligence technologies will once again be a major storyline. The widespread fear that AI would be used to manipulate the 2024 U.S.…

  • Daniel Miessler on the AI Attack/Defense Balance

    Daniel Miessler on the AI Attack/Defense Balance His conclusion: Context wins Basically whoever can see the most about the target, and can hold that picture in their mind the best, will be best at finding the vulnerabilities the fastest and taking advantage of them. Or, as the defender, applying patches or mitigations the fastest. And…

  • Use of Generative AI in Scams

    Use of Generative AI in Scams New report: “Scam GPT: GenAI and the Automation of Fraud.” This primer maps what we currently know about generative AI’s role in scams, the communities most at risk, and the broader economic and cultural shifts that are making people more willing to take risks, more vulnerable to deception, and…

  • Smashing Security podcast #437: Salesforce’s trusted domain of doom

    Smashing Security podcast #437: Salesforce’s trusted domain of doom Researchers uncovered a security flaw in Salesforce’s shiny new Agentforce. The vulnerability, dubbed “ForcedLeak”, let them smuggle AI-read instructions in via humble Web-to-Lead form… and ended up spilling data for the low, low price of five dollars. And we discuss why data breach communicationss still default…

  • The AI Fix #70: AI behaves… until it knows you’re watching

    The AI Fix #70: AI behaves… until it knows you’re watching In episode 70 of The AI Fix, our hosts learn that AI makes people more dishonest, Waymo’s robo-cars save lives but get outsmarted by a bathroom mirror, a “rescue” bot slurps up victims head-first, and China shows off a fusion robot arm that can…

  • Abusing Notion’s AI Agent for Data Theft

    Abusing Notion’s AI Agent for Data Theft Notion just released version 3.0, complete with AI agents. Because the system contains Simon Willson’s lethal trifecta, it’s vulnerable to data theft though prompt injection. First, the trifecta: The lethal trifecta of capabilities is: Access to your private data—one of the most common purposes of tools in the…

  • The AI Fix #69: How we really use ChatGPT, and will AI agents crash the economy?

    The AI Fix #69: How we really use ChatGPT, and will AI agents crash the economy? In episode 69 of The AI Fix, our hosts discover brain rot, a shark wears trainers on its fins, an AI writes a terrible J-Pop song, Graham learns that ants don’t care about AI, Mark predicts the precise date…

  • The AI Fix #68: AI telepathy, and rights for robots

    The AI Fix #68: AI telepathy, and rights for robots In episode 68 of The AI Fix, our hosts open the show by launching the thing nobody asked for but everybody wanted: our shiny new merch store – yes, including the “Would YOU trust a pigeon???” t-shirt for when you need fashion alongside health and…

  • Smashing Security podcast #434: Whopper Hackers, and AI Whoppers

    Smashing Security podcast #434: Whopper Hackers, and AI Whoppers Ever wondered what would happen if Burger King left the keys to the kingdom lying around for anyone to use? Ethical hackers did – and uncovered drive-thru recordings, hard-coded passwords, and even the power to open a Whopper outlet on the moon. Meanwhile, over in Silicon…

  • The AI Fix #67: Will Smith’s AI crowd scandal, and gullible agents fall for scams

    The AI Fix #67: Will Smith’s AI crowd scandal, and gullible agents fall for scams In episode 67 of The AI Fix, Graham talks to an AI with a fax machine, Bill Gates says there’s one job AI will never replace, criminals use Claude Code for cyberattacks, Mark reveals why GPT-5 was better than you…

  • AI in Government

    AI in Government Just a few months after Elon Musk’s retreat from his unofficial role leading the Department of Government Efficiency (DOGE), we have a clearer picture of his vision of government powered by artificial intelligence, and it has a lot more to do with consolidating power than benefitting the public. Even so, we must…

  • Top 10 Best AI Penetration Testing Companies in 2025

    Top 10 Best AI Penetration Testing Companies in 2025 AI is no longer just a buzzword; it’s a fundamental part of business operations, from customer service chatbots to complex financial models. However, this adoption has created a new and specialized attack surface. Traditional penetration testing, which focuses on network and application vulnerabilities, is insufficient to…

  • GPT-4o-mini Falls for Psychological Manipulation

    GPT-4o-mini Falls for Psychological Manipulation Interesting experiment: To design their experiment, the University of Pennsylvania researchers tested 2024’s GPT-4o-mini model on two requests that it should ideally refuse: calling the user a jerk and giving directions for how to synthesize lidocaine. The researchers created experimental prompts for both requests using each of seven different persuasion…

  • Hackers Use AI Platforms to Steal Microsoft 365 Credentials in Phishing Campaign

    Hackers Use AI Platforms to Steal Microsoft 365 Credentials in Phishing Campaign Cybercriminals are increasingly exploiting the trust organizations place in artificial intelligence platforms to conduct sophisticated phishing attacks, according to a new report from cybersecurity firm Cato Networks. The company’s Managed Detection and Response (MDR) service recently uncovered a campaign where threat actors leveraged…

  • Generative AI as a Cybercrime Assistant

    Generative AI as a Cybercrime Assistant Anthropic reports on a Claude user: We recently disrupted a sophisticated cybercriminal that used Claude Code to commit large-scale theft and extortion of personal data. The actor targeted at least 17 distinct organizations, including in healthcare, the emergency services, and government and religious institutions. Rather than encrypt the stolen…

  • Smashing Security podcast #433: How hackers turned AI into their new henchman

    Smashing Security podcast #433: How hackers turned AI into their new henchman Your AI reads the small print, and that’s a problem. This week in episode 433 of “Smashing Security” we dig into LegalPwn – malicious instructions tucked into code comments and disclaimers that sweet-talks AI into rubber-stamping dangerous payloads (or even pretending they’re a…

  • Indirect Prompt Injection Attacks Against LLM Assistants

    Indirect Prompt Injection Attacks Against LLM Assistants Really good research on practical attacks against LLM agents. “Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous” Abstract: The growing integration of LLMs into applications has introduced new security risks, notably known as Promptware­—maliciously engineered prompts designed to manipulate LLMs…

  • The AI Fix #66: OpenAI and Anthropic test each other, and everyone fails the apocalypse test

    The AI Fix #66: OpenAI and Anthropic test each other, and everyone fails the apocalypse test In episode 66 of The AI Fix, ChatGPT gives Mark and Graham a terrible lesson in anatomy, boffins at Stanford ruin sushi, Google Gemini has a self-loathing meltdown, DeepSeek gets an “F” in stopping existential threats to humanity, a…

  • We Are Still Unable to Secure LLMs from Malicious Inputs

    We Are Still Unable to Secure LLMs from Malicious Inputs Nice indirect prompt injection attack: Bargury’s attack starts with a poisoned document, which is shared to a potential victim’s Google Drive. (Bargury says a victim could have also uploaded a compromised file to their own account.) It looks like an official document on company meeting…

  • The AI Fix #65: Excel Copilot will wreck your data, and can AI fix social media?

    The AI Fix #65: Excel Copilot will wreck your data, and can AI fix social media? In episode 65 of The AI Fix, a pigeon gives a PowerPoint presentation, Mark plays Graham a song about the Transformer architecture, a robot dog delivers parcels, some robots fall over at the World Humanoid Robot Games, and Graham…

  • AI Agents Need Data Integrity

    AI Agents Need Data Integrity Think of the Web as a digital territory with its own social contract. In 2014, Tim Berners-Lee called for a “Magna Carta for the Web” to restore the balance of power between individuals and institutions. This mirrors the original charter’s purpose: ensuring that those who occupy a territory have a…

  • Subverting AIOps Systems Through Poisoned Input Data

    Subverting AIOps Systems Through Poisoned Input Data In this input integrity attack against an AI system, researchers were able to fool AIOps tools: AIOps refers to the use of LLM-based agents to gather and analyze application telemetry, including system logs, performance metrics, traces, and alerts, to detect problems and then suggest or carry out corrective…

  • The AI Fix #64: AI can be vaccinated against evil, and the “Rumble in the Silicon Jungle”

    The AI Fix #64: AI can be vaccinated against evil, and the “Rumble in the Silicon Jungle” In episode 64 of The AI Fix, AI discovers new physics, a robot crab looks for love on the beaches of Portugal, the “Godfather of AI” thinks our only hope is to build motherly AI, a robot folds…

  • Eavesdropping on Phone Conversations Through Vibrations

    Eavesdropping on Phone Conversations Through Vibrations Researchers have managed to eavesdrop on cell phone voice conversations by using radar to detect vibrations. It’s more a proof of concept than anything else. The radar detector is only ten feet away, the setup is stylized, and accuracy is poor. But it’s a start. Bruce Schneier Go to…

  • LLM Coding Integrity Breach

    LLM Coding Integrity Breach Here’s an interesting story about a failure being introduced by LLM-written code. Specifically, the LLM was doing some code refactoring, and when it moved a chunk of code from one file to another it changed a “break” to a “continue.” That turned an error logging statement into an infinite loop, which…

  • AI Applications in Cybersecurity

    AI Applications in Cybersecurity There is a really great series of online events highlighting cool uses of AI in cybersecurity, titled Prompt||GTFO. Videos from the first three events are online. And here’s where to register to attend, or participate, in the fourth. Some really great stuff here. Bruce Schneier Go to bruce schneier

  • Smashing Security podcast #430: Poisoned Calendar invites, ChatGPT, and Bromide

    Smashing Security podcast #430: Poisoned Calendar invites, ChatGPT, and Bromide A poisoned Google Calendar invite that can hijack your smart home, a man is hospitalised after ChatGPT told him to season his food with… pesticide, and some thoughts on Superman’s latest cinematic outing. All this and more is discussed in the latest edition of the…

  • What is MCP Server – How it is Powering AI-Driven Cyber Defense

    What is MCP Server – How it is Powering AI-Driven Cyber Defense MCP (Model Control Plane) Server is a centralized platform that orchestrates, manages, and secures the lifecycle of AI models deployed across an organization’s infrastructure. By providing integration, management, and real-time monitoring of models, MCP servers enable enterprises to defend against sophisticated, AI-powered cyberattacks.…

  • The AI Fix #63: GPT-5 is the best AI ever, and Jim Acosta interviews a murdered teenager’s avatar

    The AI Fix #63: GPT-5 is the best AI ever, and Jim Acosta interviews a murdered teenager’s avatar In episode 63 of The AI Fix, Unitree Robotics looks to Black Mirror episode “Metalhead” for tips on marketing its new robot dog, ChatGPT is secretly running Sweden, OpenAI introduces its first open weight model since GPT-2,…

  • ChatGPT-5 Released: What’s New With the Next-Generation AI Agent

    ChatGPT-5 Released: What’s New With the Next-Generation AI Agent OpenAI has officially launched ChatGPT-5, a new generation of its AI agent that introduces a sophisticated, unified system designed to be faster, more intelligent, and significantly more useful for real-world applications. This release marks a significant evolution from its predecessors, offering a suite of models tailored…

  • Sophos AI at Black Hat USA ’25: Anomaly detection betrayed us, so we gave it a new job

    Sophos AI at Black Hat USA ’25: Anomaly detection betrayed us, so we gave it a new job Following on from our preview, here’s Ben Gelman and Sean Bergeron’s research on enhancing command line classification with benign anomalous data Matt Wixey Go to sophos

  • China Accuses Nvidia of Putting Backdoors into Their Chips

    China Accuses Nvidia of Putting Backdoors into Their Chips The government of China has accused Nvidia of inserting a backdoor into their H20 chips: China’s cyber regulator on Thursday said it had held a meeting with Nvidia over what it called “serious security issues” with the company’s artificial intelligence chips. It said US AI experts…

  • The AI Fix #62: AI robots can now pass CAPTCHAs, and punch you in the face

    The AI Fix #62: AI robots can now pass CAPTCHAs, and punch you in the face In episode 62 of The AI Fix, your hosts learn how AI models smash through CAPTCHA roadblocks like they’re made of wet tissue paper – so much for humanity’s last line of defence. Meanwhile, we meet a bottle-flipping robot…

  • The AI Fix #61: Replit panics, deletes $1M project; AI gets gold at Math Olympiad

    The AI Fix #61: Replit panics, deletes $1M project; AI gets gold at Math Olympiad In episode 61 of The AI Fix, a robot called DeREK goes bananas, OpenAI, Google DeepMind, and Anthropic warn we may lose the ability to see what AI is thinking, a dextrous robot changes its own batteries, the USA unveils…

  • Subliminal Learning in AIs

    Subliminal Learning in AIs Today’s freaky LLM behavior: We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a “student” model learns to prefer owls when trained on sequences of numbers generated by a “teacher” model that prefers owls. This same…

  • The AI Fix #60: Elon’s AI girlfriend, the arsonist red panda, and the AI that will kill you

    The AI Fix #60: Elon’s AI girlfriend, the arsonist red panda, and the AI that will kill you In episode 60 of The AI Fix, we learn why Grok might be Elon Musk’s bid for digital immortality, how Meta is building a Manhattan-sized data centre called Prometheus, how AI is helping create carbon-sucking concrete, and…

  • Smashing Security podcast #426: Choo Choo Choose to ignore the vulnerability

    Smashing Security podcast #426: Choo Choo Choose to ignore the vulnerability In episode 426 of the “Smashing Security” podcast, Graham reveals how you can hijack a train’s brakes from 150 miles away using kit cheaper than a second-hand PlayStation. Meanwhile, Carole investigates how Grok went berserk, which didn’t stop the Department of Defense signing a…

  • The AI Fix #59: Grok thinks it’s Mecha Hitler, and AIs can think strategically

    The AI Fix #59: Grok thinks it’s Mecha Hitler, and AIs can think strategically In episode 59 of The AI Fix, our hosts ponder whether AIs need a “disagreement dial”, Mark wonders what he could do with an AI-powered “drug design engine”, Graham plays Wolfenstein instead of working, a robot graduates from high school, and…

  • The AI Fix #58: An AI runs a shop into the ground, and AI’s obsession with the number 27

    The AI Fix #58: An AI runs a shop into the ground, and AI’s obsession with the number 27 In episode 58 of “The AI Fix” podcast, our hosts discover a pair of AI headphones that don’t electrocute you, Microsoft invents “medical superintelligence”, Chucky opens a hotel, some robot footballers fall over, Jony Ive invents…

  • The AI Fix #57: AI is the best hacker in the USA, and self-learning AI

    The AI Fix #57: AI is the best hacker in the USA, and self-learning AI In episode 57 of The AI Fix, our hosts discover an AI “dream recorder”, Mark Zuckerberg tantalises OpenAI staff with $100 million signing bonuses, Graham finds out why robot butlers sit in chairs, Wikipedia holds the line against AI slop,…

  • Using AI to identify cybercrime masterminds

    Using AI to identify cybercrime masterminds Analyzing dark web forums to identify key experts on e-crime gallagherseanm Go to sophos

  • The Age of Integrity

    The Age of Integrity We need to talk about data integrity. Narrowly, the term refers to ensuring that data isn’t tampered with, either in transit or in storage. Manipulating account balances in bank databases, removing entries from criminal records, and murder by removing notations about allergies from medical records are all integrity attacks. More broadly,…

  • What LLMs Know About Their Users

    What LLMs Know About Their Users Simon Willison talks about ChatGPT’s new memory dossier feature. In his explanation, he illustrates how much the LLM—and the company—knows about its users. It’s a big quote, but I want you to read it all. Here’s a prompt you can use to give you a solid idea of what’s…

  • The AI Fix #56: ChatGPT traps man in a cult of one, and AI is actually stupid

    The AI Fix #56: ChatGPT traps man in a cult of one, and AI is actually stupid In episode 56 of The AI Fix, Anthropic and Apple have a bar fight, a woman describes her husband falling in love with ChatGPT as “not ideal”, WhatsApp’s AI helper isn’t helpful, Graham serenades a pack of headless…

  • Hackers Exploit Atlassian’s Model Context Protocol by Submitting a Malicious Support Ticket

    Hackers Exploit Atlassian’s Model Context Protocol by Submitting a Malicious Support Ticket A sophisticated attack vector targeting Atlassian’s Model Context Protocol (MCP) that allows external threat actors to gain privileged access to internal systems through malicious support tickets.  The attack, dubbed “Living off AI,” exploits the trust boundary between external users submitting support requests and…

  • Where AI Provides Value

    Where AI Provides Value If you’ve worried that AI might take your job, deprive you of your livelihood, or maybe even replace your role in society, it probably feels good to see the latest AI tools fail spectacularly. If AI recommends glue as a pizza topping, then you’re safe for another day. But the fact…

  • The AI Fix #55: Atari beats ChatGPT at chess, and Apple says AI “thinking” is an illusion

    The AI Fix #55: Atari beats ChatGPT at chess, and Apple says AI “thinking” is an illusion In episode 55 of The AI Fix, Gemini thinks a little meth won’t hurt, Mark realises what a terrifying 45mph “robot bird” is really for, Graham finds a surprising number of TikTokers in the bible, an AI discovers…

  • Hackers Advertising New Blackhat Tool Nytheon AI on Popular Hacking Forums

    Hackers Advertising New Blackhat Tool Nytheon AI on Popular Hacking Forums A sophisticated new threat platform, Nytheon AI, has emerged, which combines multiple uncensored large language models (LLMs) built specifically for malicious activities. The platform, discovered by Cato CTRL, is being actively promoted on popular hacking forums, including XSS and various Telegram channels, representing a…

  • The AI Fix #54: Will AI collapse under its own garbage, and AI charity “Hunger Games”

    The AI Fix #54: Will AI collapse under its own garbage, and AI charity “Hunger Games” In episode 54 of The AI Fix, Graham saves humanity with a CAPTCHA, Mark wonders whether AI can suffer, ChatGPT throws shade at Abba’s Björn Ulvaeus, an AI called Jack ask if you want fries with that, an artist…

  • The AI Fix #53: An AI uses blackmail to save itself, and threats make AIs work better

    The AI Fix #53: An AI uses blackmail to save itself, and threats make AIs work better In episode 53 of The AI Fix, our hosts suspect the CEO of Duolingo has been kidnapped by an AI, Sergey Brin says AIs work better if you threaten them with physical violence, Graham wonders how you put…

  • The AI Fix #52: AI adopts its own social norms, and AI DJ creates diversity scandal

    The AI Fix #52: AI adopts its own social norms, and AI DJ creates diversity scandal In episode 52 of The AI Fix, our hosts watch a non-existent musical about garlic bread, Graham shares a summer reading list of books that don’t exist, Mark feels nauseous after watching a video of Sam Altman and Jony…

  • Signal Blocks Windows Recall

    Signal Blocks Windows Recall This article gives a good rundown of the security risks of Windows Recall, and the repurposed copyright protection took that Signal used to block the AI feature from scraping Signal data. Bruce Schneier Go to bruce schneier

  • More AIs Are Taking Polls and Surveys

    More AIs Are Taking Polls and Surveys I already knew about the declining response rate for polls and surveys. The percentage of AI bots that respond to surveys is also increasing. Solutions are hard: 1. Make surveys less boring. We need to move past bland, grid-filled surveys and start designing experiences people actually want to…

  • The AI Fix #51: Divorce by coffee grounds, and why AI robots need your brain

    The AI Fix #51: Divorce by coffee grounds, and why AI robots need your brain In episode 51 of The AI Fix, a Greek man’s marriage is destroyed after ChatGPT reads his coffee, a woman dumps her husband to marry an AI called Leo, and Graham wonders whether it’s time to upload his brain into…

  • The AI Fix nominated for top podcast award. Vote now!

    The AI Fix nominated for top podcast award. Vote now! Bloomin’ eck! I’m delighted to share with you that “The AI Fix” is up for an award! Graham Cluley Go to grahamcluley

  • AI-Generated Law

    AI-Generated Law On April 14, Dubai’s ruler, Sheikh Mohammed bin Rashid Al Maktoum, announced that the United Arab Emirates would begin using artificial intelligence to help write its laws. A new Regulatory Intelligence Office would use the technology to “regularly suggest updates” to the law and “accelerate the issuance of legislation by up to 70%.” AI would create a…

  • The AI Fix #50: AI brings dead man back for killer’s trial, and the judge loves it

    The AI Fix #50: AI brings dead man back for killer’s trial, and the judge loves it In episode 50 of The AI Fix, AI brings a slain man back from the dead so he can appear at his killer’s trial, Mark gets a mysterious phone call, Trump uses AI to become Pope Donald the…

  • Chinese AI Submersible

    Chinese AI Submersible A Chinese company has developed an AI-piloted submersible that can reach speeds “similar to a destroyer or a US Navy torpedo,” dive “up to 60 metres underwater,” and “remain static for more than a month, like the stealth capabilities of a nuclear submarine.” In case you’re worried about the military applications of…

  • Fake Student Fraud in Community Colleges

    Fake Student Fraud in Community Colleges Reporting on the rise of fake students enrolling in community college courses: The bots’ goal is to bilk state and federal financial aid money by enrolling in classes, and remaining enrolled in them, long enough for aid disbursements to go out. They often accomplish this by submitting AI-generated work.…

  • The AI Fix #49: The typo from hell

    The AI Fix #49: The typo from hell In episode 49 of The AI Fix, OpenAI kills off a sycophantic bot, our hosts are introduced to a prophetic Bosnian rock band, Meta puts an electric fence around its llamas, Mark reveals he’s never tried covering a robot with olive oil, and Graham leaves a stern…

  • Another Move in the Deepfake Creation/Detection Arms Race

    Another Move in the Deepfake Creation/Detection Arms Race Deepfakes are now mimicking heartbeats In a nutshell Recent research reveals that high-quality deepfakes unintentionally retain the heartbeat patterns from their source videos, undermining traditional detection methods that relied on detecting subtle skin color changes linked to heartbeats. The assumption that deepfakes lack physiological signals, such as…

  • Applying Security Engineering to Prompt Injection Security

    Applying Security Engineering to Prompt Injection Security This seems like an important advance in LLM security against prompt injection: Google DeepMind has unveiled CaMeL (CApabilities for MachinE Learning), a new approach to stopping prompt-injection attacks that abandons the failed strategy of having AI models police themselves. Instead, CaMeL treats language models as fundamentally untrusted components…

  • The AI Fix #48: AI Jesus, and is the AI Singularity almost upon us?

    The AI Fix #48: AI Jesus, and is the AI Singularity almost upon us? In episode 48 of The AI Fix, OpenAI releases the first AI models capable of novel scientific discoveries, ChatGPT users are sick of its relentlessly positive tone, our hosts say “Alexa” a lot, OpenAI eyes a social network of its own,…

  • Regulating AI Behavior with a Hypervisor

    Regulating AI Behavior with a Hypervisor Interesting research: “Guillotine: Hypervisors for Isolating Malicious AIs.” Abstract:As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this risk, we propose Guillotine, a hypervisor architecture for sandboxing powerful AI models—models that, by accident…

  • The AI Fix #47: An AI is the best computer programmer in the world

    The AI Fix #47: An AI is the best computer programmer in the world In episode 47 of The AI Fix, o3 becomes the best competitive programmer in the world, hacked California crosswalks speak with the voice of Elon Musk and Mark Zuckerberg, Meta introduces a herd of Llamas, Graham explains what a “lollipop lady”…

  • Slopsquatting

    Slopsquatting As AI coding assistants invent nonexistent software libraries to download and use, enterprising attackers create and upload libraries with those names—laced with malware, of course. Bruce Schneier Go to bruce schneier

  • The AI Fix #46: AI can read minds now, and is your co-host a clone?

    The AI Fix #46: AI can read minds now, and is your co-host a clone? In episode 46 of The AI Fix, China trolls US tariffs, a microscopic pogoing flea-bot makes a tiny leap forward for robotics, Google unveils the Agent2Agent protocol, a robot dog is so cute it ruins Graham’s entire day, and Europe…

  • AI Vulnerability Finding

    AI Vulnerability Finding Microsoft is reporting that its AI systems are able to find new vulnerabilities in source code: Microsoft discovered eleven vulnerabilities in GRUB2, including integer and buffer overflows in filesystem parsers, command flaws, and a side-channel in cryptographic comparison. Additionally, 9 buffer overflows in parsing SquashFS, EXT4, CramFS, JFFS2, and symlinks were discovered…

  • Reimagining Democracy

    Reimagining Democracy Imagine that all of us—all of society—have landed on some alien planet and need to form a government: clean slate. We do not have any legacy systems from the United States or any other country. We do not have any special or unique interests to perturb our thinking. How would we govern ourselves?…

  • The AI Fix #45: The Turing test falls to GPT-4.5

    The AI Fix #45: The Turing test falls to GPT-4.5 In episode 45 of The AI Fix, our hosts discover that ChatGPT is running the world, Mark learns that mattress companies have scientists, Gen Z has nightmares about AI, OpenAI gets a bag, Graham eats too many cheese sandwiches, and too much training makes AIs…

  • The AI Fix #44: AI-generated malware, and a stunning AI breakthrough

    The AI Fix #44: AI-generated malware, and a stunning AI breakthrough In episode 44 of The AI Fix, ChatGPT won’t build a crystal meth lab, GPT-4o improves the show’s podcast art, some students manage to screw in a lightbulb, Google releases Gemini 2.5 Pro Experimental and nobody notices, and Mark invents a clock for measuring…

  • AIs as Trusted Third Parties

    AIs as Trusted Third Parties This is a truly fascinating paper: “Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography.” The basic idea is that AIs can act as trusted third parties: Abstract: We often interact with untrusted parties. Prioritization of privacy can limit the effectiveness of these interactions, as achieving…

  • Smashing Security podcast #410: Unleash the AI bot army against the scammers – now!

    Smashing Security podcast #410: Unleash the AI bot army against the scammers – now! A YouTuber has unleashed an innovative AI bot army to disrupt and outwit the world of online scammers, and a New York Times investigation looks into the intricate web of global money laundering. All this and more is discussed in the…

  • AI Data Poisoning

    AI Data Poisoning Cloudflare has a new feature—available to free users as well—that uses AI to generate random pages to feed to AI web crawlers: Instead of simply blocking bots, Cloudflare’s new system lures them into a “maze” of realistic-looking but irrelevant pages, wasting the crawler’s computing resources. The approach is a notable shift from…

  • The AI Fix #43: I, for one, welcome our new robot overlords!

    The AI Fix #43: I, for one, welcome our new robot overlords! In episode 43 of The AI Fix, our hosts discover a robot that isn’t terrifying, a newspaper shuns journalists in favour of AI, Graham watches a robot dog learn to stand, an AI computer programmer develops a familiar attitude, and New York tries…

  • My Writings Are in the LibGen AI Training Corpus

    My Writings Are in the LibGen AI Training Corpus The Atlantic has a search tool that allows you to search for specific works in the “LibGen” database of copyrighted works that Meta used to train its AI models. (The rest of the article is behind a paywall, but not the search tool.) It’s impossible to…