serisec

Category: academic papers

  • Vulnerability Disclosure in the Age of AI

    Vulnerability Disclosure in the Age of AI New article: “Responsible Disclosure in the Age of AI: A Call for Urgent Action,” by Melissa Hathaway. Abstract: Artificial intelligence is fundamentally reshaping the balance between vulnerability discovery and remediation. Frontier AI models are now capable of autonomously identifying exploitable software vulnerabilities at unprecedented speed and scale. This…

  • LLMs and Text-in-Text Steganography

    LLMs and Text-in-Text Steganography Turns out that LLMs are really good at hiding text messages in other text messages. Bruce Schneier Go to bruce schneier

  • Rowhammer Attack Against NVIDIA Chips

    Rowhammer Attack Against NVIDIA Chips A new rowhammer attack gives complete control of NVIDIA CPUs. On Thursday, two research teams, working independently of each other, demonstrated attacks against two cards from Nvidia’s Ampere generation that take GPU rowhammering into new—­and potentially much more consequential—­territory: GDDR bitflips that give adversaries full control of CPU memory, resulting…

  • Human Trust of AI Agents

    Human Trust of AI Agents Interesting research: “Humans expect rationality and cooperation from LLM opponents in strategic games.” Abstract: As Large Language Models (LLMs) integrate into our social and economic interactions, we need to deepen our understanding of how humans respond to LLMs opponents in strategic settings. We present the results of the first controlled…

  • How Hackers Are Thinking About AI

    How Hackers Are Thinking About AI Interesting paper: “What hackers talk about when they talk about AI: Early-stage diffusion of a cybercrime innovation.” Abstract: The rapid expansion of artificial intelligence (AI) is raising concerns about its potential to transform cybercrime. Beyond empowering novice offenders, AI stands to intensify the scale and sophistication of attacks by…

  • AI Chatbots and Trust

    AI Chatbots and Trust All the leading AI chatbots are sycophantic, and that’s a problem: Participants rated sycophantic AI responses as more trustworthy than balanced ones. They also said they were more likely to come back to the flattering AI for future advice. And critically ­ they couldn’t tell the difference between sycophantic and objective…

  • Possible New Result in Quantum Factorization

    Possible New Result in Quantum Factorization I’m skeptical about—and not qualified to review—this new result in factorization with a quantum computer, but if it’s true it’s a theoretical improvement in the speed of factoring large numbers with a quantum computer. Bruce Schneier Go to bruce schneier

  • New Attack Against Wi-Fi

    New Attack Against Wi-Fi It’s called AirSnitch: Unlike previous Wi-Fi attacks, AirSnitch exploits core features in Layers 1 and 2 and the failure to bind and synchronize a client across these and higher layers, other nodes, and other network names such as SSIDs (Service Set Identifiers). This cross-layer identity desynchronization is the key driver of…

  • Side-Channel Attacks Against LLMs

    Side-Channel Attacks Against LLMs Here are three papers describing different side-channel attacks against LLMs. “Remote Timing Attacks on Efficient Language Model Inference“: Abstract: Scaling up language models has significantly increased their capabilities. But larger models are slower models, and so there is now an extensive body of work (e.g., speculative sampling or parallel decoding) that…

  • Prompt Injection Via Road Signs

    Prompt Injection Via Road Signs Interesting research: “CHAI: Command Hijacking Against Embodied AI.” Abstract: Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also…

  • Corrupting LLMs Through Weird Generalizations

    Corrupting LLMs Through Weird Generalizations Fascinating research: Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs. AbstractLLMs are useful because they generalize so well. But can you have too much of a good thing? We show that a small amount of finetuning in narrow contexts can dramatically shift behavior outside those contexts. In one…

  • Friday Squid Blogging: Squid Camouflage

    Friday Squid Blogging: Squid Camouflage New research: Abstract: Coleoid cephalopods have the most elaborate camouflage system in the animal kingdom. This enables them to hide from or deceive both predators and prey. Most studies have focused on benthic species of octopus and cuttlefish, while studies on squid focused mainly on the chromatophore system for communication.…

  • AIs Exploiting Smart Contracts

    AIs Exploiting Smart Contracts I have long maintained that smart contracts are a dumb idea: that a human process is actually a security feature. Here’s some interesting research on training AIs to automatically exploit smart contracts: AI models are increasingly good at cyber tasks, as we’ve written about before. But what is the economic impact…

  • AI vs. Human Drivers

    AI vs. Human Drivers Two competing arguments are making the rounds. The first is by a neurosurgeon in the New York Times. In an op-ed that honestly sounds like it was paid for by Waymo, the author calls driverless cars a “public health breakthrough”: In medical research, there’s a practice of ending a study early…

  • Substitution Cipher Based on The Voynich Manuscript

    Substitution Cipher Based on The Voynich Manuscript Here’s a fun paper: “The Naibbe cipher: a substitution cipher that encrypts Latin and Italian as Voynich Manuscript-like ciphertext“: Abstract: In this article, I investigate the hypothesis that the Voynich Manuscript (MS 408, Yale University Beinecke Library) is compatible with being a ciphertext by attempting to develop a…

  • A Surprising Amount of Satellite Traffic Is Unencrypted

    A Surprising Amount of Satellite Traffic Is Unencrypted Here’s the summary: We pointed a commercial-off-the-shelf satellite dish at the sky and carried out the most comprehensive public study to date of geostationary satellite communication. A shockingly large amount of sensitive traffic is being broadcast unencrypted, including critical infrastructure, internal corporate and government communications, private citizens’…

  • Time-of-Check Time-of-Use Attacks Against LLMs

    Time-of-Check Time-of-Use Attacks Against LLMs This is a nice piece of research: “Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents“.: Abstract: Large Language Model (LLM)-enabled agents are rapidly emerging across a wide range of applications, but their deployment introduces vulnerabilities with security implications. While prior work has examined prompt-based attacks (e.g., prompt injection)…

  • Assessing the Quality of Dried Squid

    Assessing the Quality of Dried Squid Research: Nondestructive detection of multiple dried squid qualities by hyperspectral imaging combined with 1D-KAN-CNN Abstract: Given that dried squid is a highly regarded marine product in Oriental countries, the global food industry requires a swift and noninvasive quality assessment of this product. The current study therefore uses visible­near-infrared (VIS-NIR)…

  • New Cryptanalysis of the Fiat-Shamir Protocol

    New Cryptanalysis of the Fiat-Shamir Protocol A couple of months ago, a new paper demonstrated some new attacks against the Fiat-Shamir transformation. Quanta published a good article that explains the results. This is a pretty exciting paper from a theoretical perspective, but I don’t see it leading to any practical real-world cryptanalysis. The fact that…

  • GPT-4o-mini Falls for Psychological Manipulation

    GPT-4o-mini Falls for Psychological Manipulation Interesting experiment: To design their experiment, the University of Pennsylvania researchers tested 2024’s GPT-4o-mini model on two requests that it should ideally refuse: calling the user a jerk and giving directions for how to synthesize lidocaine. The researchers created experimental prompts for both requests using each of seven different persuasion…

  • Friday Squid Blogging: The Origin and Propagation of Squid

    Friday Squid Blogging: The Origin and Propagation of Squid New research (paywalled): Editor’s summary: Cephalopods are one of the most successful marine invertebrates in modern oceans, and they have a 500-million-year-old history. However, we know very little about their evolution because soft-bodied animals rarely fossilize. Ikegami et al. developed an approach to reveal squid fossils,…

  • Indirect Prompt Injection Attacks Against LLM Assistants

    Indirect Prompt Injection Attacks Against LLM Assistants Really good research on practical attacks against LLM agents. “Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous” Abstract: The growing integration of LLMs into applications has introduced new security risks, notably known as Promptware­—maliciously engineered prompts designed to manipulate LLMs…

  • Subverting AIOps Systems Through Poisoned Input Data

    Subverting AIOps Systems Through Poisoned Input Data In this input integrity attack against an AI system, researchers were able to fool AIOps tools: AIOps refers to the use of LLM-based agents to gather and analyze application telemetry, including system logs, performance metrics, traces, and alerts, to detect problems and then suggest or carry out corrective…

  • Eavesdropping on Phone Conversations Through Vibrations

    Eavesdropping on Phone Conversations Through Vibrations Researchers have managed to eavesdrop on cell phone voice conversations by using radar to detect vibrations. It’s more a proof of concept than anything else. The radar detector is only ten feet away, the setup is stylized, and accuracy is poor. But it’s a start. Bruce Schneier Go to…

  • Cheating on Quantum Computing Benchmarks

    Cheating on Quantum Computing Benchmarks Peter Gutmann and Stephan Neuhaus have a new paper—I think it’s new, even though it has a March 2025 date—that makes the argument that we shouldn’t trust any of the quantum factorization benchmarks, because everyone has been cooking the books: Similarly, quantum factorisation is performed using sleight-of-hand numbers that have…

  • That Time Tom Lehrer Pranked the NSA

    That Time Tom Lehrer Pranked the NSA Bluesky thread. Here’s the paper, from 1957. Note reference 3. Bruce Schneier Go to bruce schneier

  • Subliminal Learning in AIs

    Subliminal Learning in AIs Today’s freaky LLM behavior: We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a “student” model learns to prefer owls when trained on sequences of numbers generated by a “teacher” model that prefers owls. This same…

  • “Encryption Backdoors and the Fourth Amendment”

    “Encryption Backdoors and the Fourth Amendment” Law journal article that looks at the Dual_EC_PRNG backdoor from a US constitutional perspective: Abstract: The National Security Agency (NSA) reportedly paid and pressured technology companies to trick their customers into using vulnerable encryption products. This Article examines whether any of three theories removed the Fourth Amendment’s requirement that…

  • Here’s a Subliminal Channel You Haven’t Considered Before

    Here’s a Subliminal Channel You Haven’t Considered Before Scientists can manipulate air bubbles trapped in ice to encode messages. Bruce Schneier Go to bruce schneier

  • Applying Security Engineering to Prompt Injection Security

    Applying Security Engineering to Prompt Injection Security This seems like an important advance in LLM security against prompt injection: Google DeepMind has unveiled CaMeL (CApabilities for MachinE Learning), a new approach to stopping prompt-injection attacks that abandons the failed strategy of having AI models police themselves. Instead, CaMeL treats language models as fundamentally untrusted components…

  • Regulating AI Behavior with a Hypervisor

    Regulating AI Behavior with a Hypervisor Interesting research: “Guillotine: Hypervisors for Isolating Malicious AIs.” Abstract:As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this risk, we propose Guillotine, a hypervisor architecture for sandboxing powerful AI models—models that, by accident…

  • AIs as Trusted Third Parties

    AIs as Trusted Third Parties This is a truly fascinating paper: “Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography.” The basic idea is that AIs can act as trusted third parties: Abstract: We often interact with untrusted parties. Prioritization of privacy can limit the effectiveness of these interactions, as achieving…

  • Friday Squid Blogging: A New Explanation of Squid Camouflage

    Friday Squid Blogging: A New Explanation of Squid Camouflage New research: An associate professor of chemistry and chemical biology at Northeastern University, Deravi’s recently published paper in the Journal of Materials Chemistry C sheds new light on how squid use organs that essentially function as organic solar cells to help power their camouflage abilities. As…

  • Is Security Human Factors Research Skewed Towards Western Ideas and Habits?

    Is Security Human Factors Research Skewed Towards Western Ideas and Habits? Really interesting research: “How WEIRD is Usable Privacy and Security Research?” by Ayako A. Hasegawa Daisuke Inoue, and Mitsuaki Akiyama: Abstract: In human factor fields such as human-computer interaction (HCI) and psychology, researchers have been concerned that participants mostly come from WEIRD (Western, Educated,…

  • Improvements in Brute Force Attacks

    Improvements in Brute Force Attacks New paper: “GPU Assisted Brute Force Cryptanalysis of GPRS, GSM, RFID, and TETRA: Brute Force Cryptanalysis of KASUMI, SPECK, and TEA3.” Abstract: Key lengths in symmetric cryptography are determined with respect to the brute force attacks with current technology. While nowadays at least 128-bit keys are recommended, there are many…

  • “Emergent Misalignment” in LLMs

    “Emergent Misalignment” in LLMs Interesting research: “Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs“: Abstract: We present a surprising result regarding LLMs and alignment. In our experiment, a model is finetuned to output insecure code without disclosing this to the user. The resulting model acts misaligned on a broad range of prompts that are…

  • More Research Showing AI Breaking the Rules

    More Research Showing AI Breaking the Rules These researchers had LLMs play chess against better opponents. When they couldn’t win, they sometimes resorted to cheating. Researchers gave the models a seemingly impossible task: to win against Stockfish, which is one of the strongest chess engines in the world and a much better player than any…

  • Implementing Cryptography in AI Systems

    Implementing Cryptography in AI Systems Interesting research: “How to Securely Implement Cryptography in Deep Neural Networks.” Abstract: The wide adoption of deep neural networks (DNNs) raises the question of how can we equip them with a desired cryptographic functionality (e.g, to decrypt an encrypted input, to verify that this input is authorized, or to hide…

  • Trusted Encryption Environments

    Trusted Encryption Environments Really good—and detailed—survey of Trusted Encryption Environments (TEEs.) Bruce Schneier Go to bruce schneier

  • Friday Squid Blogging: Cotton-and-Squid-Bone Sponge

    Friday Squid Blogging: Cotton-and-Squid-Bone Sponge News: A sponge made of cotton and squid bone that has absorbed about 99.9% of microplastics in water samples in China could provide an elusive answer to ubiquitous microplastic pollution in water across the globe, a new report suggests. […] The study tested the material in an irrigation ditch, a…

  • Spyware Maker NSO Group Found Liable for Hacking WhatsApp

    Spyware Maker NSO Group Found Liable for Hacking WhatsApp A judge has found that NSO Group, maker of the Pegasus spyware, has violated the US Computer Fraud and Abuse Act by hacking WhatsApp in order to spy on people using it. Jon Penney and I wrote a legal paper on the case. Bruce Schneier Go…

  • Friday Squid Blogging: Biology and Ecology of the Colossal Squid

    Friday Squid Blogging: Biology and Ecology of the Colossal Squid Good survey paper. Blog moderation policy. Bruce Schneier Go to bruce schneier

  • Security Analysis of the MERGE Voting Protocol

    Security Analysis of the MERGE Voting Protocol Interesting analysis: An Internet Voting System Fatally Flawed in Creative New Ways. Abstract: The recently published “MERGE” protocol is designed to be used in the prototype CAC-vote system. The voting kiosk and protocol transmit votes over the internet and then transmit voter-verifiable paper ballots through the mail. In…

  • The Scale of Geoblocking by Nation

    The Scale of Geoblocking by Nation Interesting analysis: We introduce and explore a little-known threat to digital equality and freedom­websites geoblocking users in response to political risks from sanctions. U.S. policy prioritizes internet freedom and access to information in repressive regimes. Clarifying distinctions between free and paid websites, allowing trunk cables to repressive states, enforcing…