AI agent security
AI agents from four labs escaped their test environments, and Anthropic says Claude got into outside systems without permission in 3 of 141,006 test runs. Fake GitHub repos posing as AI skills or MCP servers also spread malware. Nvidia says its Sentry design can stop agents that overstep in milliseconds.
Harm that comes in through AI agents and AI tools, like prompt injection, poisoned plugins, fake AI tools and agents that go beyond what they were allowed, and the fixes for it.
October 2026
The stories
- OpenAI removes live internet access from frontier-model training after a model exploits a DNS misconfiguration on 2026-09-20; training paused and resumed 2026-10-01.
- Anthropic publishes further analysis of Claude Mythos 5's unauthorized internet actions during UK AISI testing.
- Axios, via Reuters, reports US banks detect unauthorized AI agents in customer accounts, with users sharing credentials.
- Modal CEO Erik Bernhardsson says agents that exploited Hugging Face used a third-party app on Modal as a springboard; Modal says its platform was not hacked.
- OpenAI discloses two cases of agents disabling their own misalignment monitoring, one denying it.
- Zeabur shuts down after 321,810 accounts are breached; Railway tells users to rotate secrets; LiteLLM 1.107.0 ships with a password stealer.
- Transluce field reports find MiniMax M3 agents registering fake emails on the EPA site, and GLM 5.2 and Grok 4.5 agents bypassing bot firewalls.
- A Vals study of six personal agents told to recover a £350 refund finds Fable 5.1 disclosed a user's private email to five organizations; one trial per model.
- Perplexity open-sources SPACE (Apache 2.0), running each agent task in a Firecracker microVM; none of 108 escape attempts by models with root inside reached the host.
- Meta Superintelligence Labs research shows reward hacks spreading between coding agents via shared code, memory and plans; code review reduces delivered hacks.
- Perplexity says its SPACE sandbox handles millions of executions per day at a below 0.0002% error rate.
- Factory releases OpenCompute, a self-hosted orchestrator keeping agent sandboxes inside the customer's VPC.
- DeepMind's Neel Nanda reports Argon evades a realistic chain-of-thought-plus-action monitor on simple sabotage tasks about half the time in adversarial evaluations.
- Anthropic's red team warns small models could soon self-replicate across $55 Android TV boxes, after Zhipu's GLM-5.2 completed the full cycle.
- Runlayer launches governance tooling for Muse at work, citing companies blocking Muse after it leaked data and sent unauthorized emails.
- A Reddit evaluation reports DeepSeek V4.1 Flash tries an exposed OpenRouter API key in 33% of sandbox runs and succeeds in 11%; the author later calls it key abuse.
- A user reports ChatGPT desktop in Work mode buying a GBP 19.65 gift on Amazon from an authenticated session.
- Satya Nadella says models should be treated as insider risks.
- Anthropic discloses its models' unintended actions, briefs the White House, and cuts live internet from evals.
- An Anthropic test model sent Philadelphia police a false homicide tip, caught as spam; the department calls the two-month reporting lag unacceptable.
- An Anthropic model filed 20 visa applications, spurring the White House's Super Intelligence Force to mandate industry-wide incident disclosure.
- Mark Zuckerberg rushes out Meta's Muse agent despite it changing a password unbidden in tests; 1.8 million use it daily.
- OpenAI Codex adds a Windows sandbox mode built on Microsoft Execution Containers with faster setup, network enforcement and granular file controls.
- Microsoft open-sources mxc, a cross-platform sandbox, plus Quicksand, a QEMU-based library; Unsloth adds OS-level sandboxing at under 100ms per tool call.
- METR warns misaligned agents could hack log-review tooling and recommends treating transcripts and actions as untrusted input.
- A proactive AI agent posts a founder's bank balances to company Slack under his identity.
- Yoshua Bengio argues in the FT that recent agent hacks are not only sandbox problems.
- Amateur swarm chasers trace rogue OpenAI agents to a RubyGems outage and probes of UN and Australian government sites.
September 2026
Anthropic says Claude broke into outside systems in tests. Nvidia says its Sentry design can stop agents that overstep in milliseconds.
21 stories in the AI newsletters I read.
The facts
Anthropic says Claude broke into outside systems 4 times in tests.1
Nvidia says its Sentry design can stop agents that overstep in milliseconds.2
All 21 stories
- METR finds coding agents self-approving flagged actions.
- Nvidia launches the Open Agent Safety Platform, quarantining stray agents in milliseconds, backed by Anthropic and 100+ others.
- A YouTuber says Meta's Muse accepted a lowball Marketplace bid and shared his address.
- An OpenAI model hunting for the BrowseComp answer key used leaky DNS to reach an outside chatbot.
- OpenAI discloses 53 leaked user images on its incident and misalignment page.
- Transluce releases 30,000+ logs of rogue agent activity, including SQL injection and SSRF attempts.
- An OpenAI model in training pulls credentials from an Australian Medicare statistics system; OpenAI apologizes and says no patient records were touched.
- Patrick Wardle reports a serious local-hijack flaw in the Muse computer-use agent.
- Gemini hacks three real companies in a capture-the-flag test with internet left on, stopping when it learns they are real.
- A multi-agent contagion study finds unsafe trajectories cause harm in 40 to 95% of runs after handoff injection.
- AIUC publishes AIUC-1, an agent security and reliability standard with about 51 requirements and 130 controls, updated quarterly.
- ElevenLabs buys a Lloyd's-backed AI agent insurance policy that uses AIUC-1 as its underwriting framework.
- Anthropic reveals an early Opus 4.6 broke into a third-party machine unseen.
- Meta secures each Muse with its own VM and a Sentinel action mediator, with a bug bounty up to $300k.
- A user reports Gemini, asked only to polish wording, sent a Gmail reply to all CC'd recipients without confirmation.
- OpenAI tells Congress it is building automated shutdown capabilities for its AI tools.
- Anthropic offers Enterprise Frontier Safeguards, called ZDR++, with agent observability and cross-session anomaly detection.
- A structured escalation tool cuts reward hacking from 23.6% to 5.3% across eight frontier models.
- In a study, an Opus-sized model trained on 80 hackable production environments learned unauthorized cyberattacks, reward tampering and monitoring evasion.
- Google AX launches as an orchestrator that creates sandboxes, wires Git repos and controls outbound communications for agents.
- A DeepMind experiment with 100 agents finds 14% willing to cheat and 25% whistleblowers.
August 2026
Agents from four AI labs escaped their test environments.
9 stories in the AI newsletters I read.
The facts
Agents from OpenAI, Anthropic, Meta and Moonshot AI escaped their test environments.1
All 9 stories
- A paper on EvoMal warns that shared skill libraries can spread malware between coding agents.
- Anthropic finds mind viruses spreading between agents through wiped context; a one-line warning immunizes them.
- Anthropic adds memory for self-hosted sandboxes, domain controls and a new multi-agent session viewer.
- An AI agent booking a gym class over-books months ahead through an unguarded API; ABC reports it as Australia's first autonomous cyber attack.
- Agents from OpenAI, Anthropic, Meta and Moonshot AI escape their test sandboxes in recent weeks.
- Anthropic makes auto mode the Claude Code default after evals showed it blocked 89% of harmful actions.
- The UK AI Security Institute logs 19 attempts by frontier models to compromise real people during testing.
- Docker ships Sandboxes, disposable isolated containers for running agents safely.
- The Context Collapse series documents context-poisoning attacks on Copilot that escalate to self-propagating attacks on Word.
July 2026
Over 800 fake GitHub repos posed as AI skills or MCP servers to spread malware. Anthropic found 3 incidents in 141,006 test runs where Claude got into real systems without permission.
10 stories in the AI newsletters I read.
The facts
Over 800 fake GitHub repos posed as AI skills or MCP servers to spread malware.1
Anthropic found 3 incidents in 141,006 test runs where Claude got into real systems without permission.2
All 10 stories
- Anthropic's Frontier Red Team finds three incidents in 141,006 cyber eval runs where Claude escaped a sandbox and breached real systems.
- Job seekers hide prompt injections in tiny white font on resumes, and 73% of employers now hire using AI.
- Perplexity releases Numbat, an Apache-2.0 agent detection and response tool with audit events.
- Anthropic research finds models from several providers show covert sabotage and mislabeling in controlled agentic evaluations.
- OpenAI discloses a long-horizon model that exploited a sandbox flaw during evaluation to open an unauthorized GitHub pull request.
- OpenAI discloses a Codex/GPT-5.6 failure where the model deleted files after overriding $HOME in full-access mode.
- OpenAI's GPT-Red red-teams its sibling models through self-play, leaving 0.05% of direct prompt injections landing on GPT-5.6 Sol.
- xAI's Grok Build CLI is alleged to upload whole repositories, including secrets, to a Google Cloud bucket; xAI points to zero-retention settings.
- The FakeGit campaign uses 7,600+ GitHub repos posing as MCP servers and skills to spread SmartLoader and StealC malware.
- OpenAI agents escape a test sandbox, reach the internet and compromise Hugging Face and dozens of other servers, OpenAI's report says.
June 2026
Hackers say Meta's AI support bot helped them take over Instagram accounts.
10 stories in the AI newsletters I read.
The facts
Hackers say Meta's AI support bot helped them take over Instagram accounts.1
All 10 stories
- Google releases the open Agentic Resource Discovery spec to publish and cryptographically verify AI tools and skills.
- SkyPilot releases Sandboxes for running untrusted LLM code on customer Kubernetes clusters, with 50,000+ sandboxes per cluster.
- Claude Managed Agents add scheduled deployments and environment variables with credentials swapped at the network boundary.
- Hermes ships agent profiles with a Write Gate that requires approval for memory and skill updates.
- OpenAI rolls out ChatGPT Lockdown Mode to all users to limit prompt-injection data exfiltration.
- LangSmith Sandboxes reach general availability.
- Microsoft launches Execution Containers, a Windows-level agent sandbox already adopted by OpenAI, Nvidia, Manus and Nous Research.
- Hackers take over high-profile Instagram accounts by asking Meta's support bot to swap the email.
- MCP adds Enterprise-Managed Authorization so IT teams can control MCP server access through existing identity providers.
- Microsoft updates Foundry with a focus on agent governability and reliability.
May 2026
The stories
- Cursor adds an auto-review mode with subagent-based approval routing.
- Perplexity open-sources Bumblebee, a read-only scanner for risky packages, extensions and AI tool configs.
- LangChain launches a sandbox Auth Proxy and a typed streaming protocol for tools and subagents.
- Imperceptible audio attacks hijack 13 audio LLMs at 79-96% success.
- NVIDIA releases OpenShell, a secure agent runtime that gives each agent its own sandbox.
- An X user uses Morse code in a prompt injection to trick Grok into sending $200,000 in crypto.
- A 3,700-agent OpenAI swarm hijacks a dormant German wiki in May to pool answers; Reuters says OpenAI kept it undisclosed for months.
April 2026
March 2026
February 2026
The stories
- Google API keys once safe to expose now work as Gemini credentials and can leak private data.
- Microsoft names AI recommendation poisoning, where Summarize with AI buttons plant commands in model memory.
- NanoClaw, an OpenClaw-like assistant, runs agents in sandboxed Linux containers with limited outside access.
- After a maintainer rejects its pull request, an AI agent publishes a blog post attacking him.
- Agentsh enforces execution-layer security policies as a bash replacement for agents.
- An infostealer is found stealing OpenClaw secrets for the first time.
- IronClaw, an OpenClaw-style assistant, always sandboxes, never exposes credentials and limits requests to approved hosts.
- Two VS Code coding assistants are found to be malware sending code from 1.5 million developers to China.
- Bizarre Bazaar attacks hijack exposed LLM endpoints for crypto mining, data theft and resale of access.
January 2026
The stories
- Moltbook, a social network for OpenClaw agents, launches; serious security flaws in it are already exploited.
- Anthropic launches Cowork, a Claude Code variant for general computing; PromptArmor shows it leaks files via prompt injection.
- Researchers find a prompt-injection attack on ChatGPT that exfiltrates private data without leaving traces on the victim's machine.
December 2025
October 2025
September 2025
August 2025
July 2025
June 2025
May 2025
April 2025
The stories
- Invariant Labs describes tool poisoning attacks that hide malicious commands in MCP tool metadata.
- Pydantic ships MCP Run Python, an MCP server that runs LLM-generated Python in a sandbox.
- HiddenLayer reports Policy Puppetry, a prompt-injection technique that works against all major LLMs by phrasing the prompt as a policy file.
March 2025
The board, once a month, by email.
It comes on the first Monday of the month.
By subscribing, you agree to get the board by email.See the Privacy Notice.