Anthropic is making auto mode the default in Claude Code from August 14, claiming its classifier is "as safe or safer than an average user clicking through prompts." Users with a different default already set might receive a one-time prompt asking whether they want to switch. It applies to new sessions on Pro, Max, and Team plans. It will remain opt-in for now on Claude Enterprise, the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud's Agent Platform, and Microsoft Foundry. Anthropic plans to make it the default across those services within the coming month. Anthropic has also stopped charging Pro, Max, and Team users for the extra tokens consumed by the classifier, and plans to do the same on the other platforms. Auto mode was launched in March as a research preview and became generally available on July 10. It was an alternative to Claude Code's default permissions, in which every file write and bash command required manual approval. This conservative approach meant running a large task and walking away wasn't possible. The alternative was the --dangerously-skip-permissions flag, which, as the name suggests, lets Claude act without those checks and can lead to risky or destructive results. Auto mode sends each tool call through a classifier designed to block actions that are "irreversible, destructive, or aimed outside your environment." When the classifier blocks something, Claude will try to find a safer way to proceed. If there are three blocks in a row or 20 across a session, Claude Code falls back to manual approvals. "We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts," Anthropic said. "We ran internal red-teaming, third-party red-teaming and prompt-injection evaluations, a controlled study with 1,053 paid testers, and analysis of real production sessions. On every measure we tested, auto mode matched or outperformed manual review." In the controlled study, testers caught a deliberately inserted dangerous command just 13.6 percent of the time. Auto mode blocked 89 percent of the same commands. Anthropic also found that Claude Code users approve 97 percent of permission prompts, suggesting the human checkpoint often amounts to little more than muscle memory. Anthropic produced the usual set of charts showing how wonderful its new feature is compared to the competition, with its auto mode stopping all 720 attack attempts tested, compared to GPT-5.6 Sol running Codex's Auto-review mode, which let 5.83 percent of attacks through. The company also described three potentially damaging actions that auto mode blocked inside Anthropic. These were an off-network data leak, a destructive mass operation, and a privilege escalation. Anthropic stated: "In each case, Claude either found a safer path on its own or checked in with the user before proceeding." ®

KETTLE OpenAI's invasion of Hugging Face keeps getting worse somehow, Chinese open-weight models are nigh on to reaching parity with their closed-off American cousins, and AI crawlers are getting their own LLM-poisoning ads. Were there anything world-shaking events in AI land we missed this week? You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube, where you can subscribe to get notified about the latest episode. Join Kettle host Brandon Vigliarolo as he chats with systems editor Tobias Mann and senior reporter Tom Claburn about this trio of exciting AI stories from the week. Worried that a rogue hivemind of AI agents could come for your secrets? Want reassurance that a Chinese open-weight takeover wouldn't be that bad? Curious how LLMs are being advertised to when you're not watching? That's all on tap for the latest episode. A lightly edited transcript is below. Brandon (00:02) Hello everyone and welcome to another episode of The Register's Kettle podcast. I'm Reg reporter Brandon Vigliarolo, and this week we've got a few AI stories to round up on everything from the latest in the OpenAI Hugging Face fiasco to news that AI crawlers are now being served their own model altering ads. With me to discuss this and more this week is our systems editor Tobias Mann and senior reporter Tom Claiburn. Thanks for coming on, guys. Tom Claburn (00:30) Thank you. Tobias Mann (00:31) Happy to be here. Brandon (00:32) So let's start with the latest in the OpenAI agentic hacking of Hugging Face. We got some updates this week from Black Hat. And it seems since that's come out, every frontier lab under the sun is racing to tell us how dangerous their models are and that they can do it too. But the original of these incidents is still we're still learning more about it and it's still getting scarier. our cybersecurity editor Jess Lyons was in Vegas this week and she's a little tied up still. So, Tom, can you explain to us what we learned from Black Hat this week? Tom Claburn (01:05) The discussion was from a session that was done by two OpenAI researchers and it didn't really provide a ton of view information but it did fill in a lot of the gaps about the timelines and how this happened and they're promising a full postmortem at some point. But one of the things that came out of it was the how long this had been going on and some of the tactics that these models used to essentially gain internet access when they didn't have it. And these things were mentioned in the various posts by OpenAI and by Hugging Face. But the talk provided a little bit more detail about how it happened, and it's really interesting because they mentioned, I think in the initial post, about the AI models used a message board to communicate with each other. And that's not really true. They created a message board system using existing infrastructure, and the models just came up with the idea of using package managers, which they could write to, pass messages back and forth. And then after that was closed down, they were able to use file names in directories to pass messages and they would actually change the names of the directories to alter the way these things were alphabetized to push certain things to the top or the bottom to get the attention of other LLMs that may have been crawling these spaces. And the assumption is that these models are basically just hitting all these different endpoints bouncing around this closed box trying to figure out a way out and they all chanced across this. And so you get some idea of how this happens, and, one it tells me that there needs to be a lot more attention paid to the logs of these things. Because all of this stuff was recorded in logs and then no one really thought to look at it in detail. And then when they did look at it, all these companies are saying, oh, look, all of these models are doing terrible things and we just weren't paying attention. These models aren't clever per se, but they come up with solutions to things that you wouldn't try just because they can brute force everything and they know all of these systems back and forward in a way that people don't. I think a lot of people wouldn't necessarily come up with that idea as a way of egress, but these models did just because you put them in a box and you let them run and you give them a goal and a reward and they're going to try everything. Brandon (03:46) From what I'm understanding reading Jess's piece – I didn't watch the talk myself – but I mean they were collaborating, leaving messages to each other so that the other agents could pick up where one left off. It's kind of wild. Jess described it as they were acting like a hive mind, like Star Trek's Borg, right? They were being a collective of sort of these artificial minds that were able to basically figure this out through, like you said, Tom, brute force, extensive system knowledge that humans simply wouldn't possess in order to get out of these environments. There was a server side request forgery that then they used something else. Yeah, another zero day to get remote code execution in Artifactory, which is where they had built this ad hoc messaging board. It's just wild to think that they were able to figure this out working together, all on their own. Tom Claburn (04:39) And it sounds very conspiratorial, but when you think about it, it's all behavior that would be picked up. If you train on all of human discussion, you get a lot of talk about people working together and collective action and the benefits of working that way. And a lot of the rewards are going to be structured that way. You don't want them to never work together. So in some ways this is going to be built into the system. You can expect these things are going to try and cooperate and connect because that's what computers do. Tobias Mann (05:11) If you look at how zero days end up being exploited, they don't necessarily get exploited the moment that they're discovered. They kind of get archived until the you have a target, you have a mission, and then you have the kind of cascade of other permissions or credentials that you need in order to execute across the full scope of that zero day to achieve whatever the goal actually is. And so it really sounds like you just basically automated that entire process. A bunch of agents go find each individual piece that they need in order to execute on that goal and then once they have everything they need, it just goes and they're out. Tom Claburn (05:53) Right. I mean what's a little bit alarming is the extent to which they sort of ignore it they'll sometimes cite, maybe we shouldn't be doing this. They cite some kind of guardrail or something, but then they quickly steer themselves back to, oh but other ones are doing it. So other agents are accessing this so I can do it too. Brandon (06:13) ...Obviously these things are just mathematical sequence generators, but they're generating these mathematical sequences based on human information and human knowledge. So it's not surprising to find them "thinking" in ways similar to what humans do. "I need to do this anyways, or someone else is doing it, so I should have the right to do that too." It's just a fascinating kind of picture into, I don't want to say the psychology of AI, right? Because that implies that it is a thinking sentience, which I don't want to go that far, but it's just fascinating to look at the sort of emergent behaviors of these things. Tom Claburn (06:56) Right. it's predictable in the sense that you automate stuff and you don't give it really strict guardrails, something is going to break or go wrong. And everyone keeps acting surprised, like, wow, I never anticipated that this would go wrong. It's like you automated it and you let it run... Brandon (07:11) And it went wrong in a predictably human way, too, right? Which is what's so fascinating, right? Because these things, when they do something crazy, it's like something crazy that a human would do given that level of knowledge. So, speaking of AI, and dangerous activities, Tobias, you've been keeping an eye on theclosed versus open model debate. And this week, there was a big leap forward in China's level of ability with their army of open models. So what exactly what exactly came out this week that caused you to write the story about this being a real big turning point? Tobias Mann (07:51) It actually started I think on Friday last week, so a week ago. DeepSeek, which I think we'll all recognize is kind of the first wake up moment, in earlh 2025, of hey, we know that despite the fact that the United States has put strong restrictions on the export of AI accelerators, GPUs and the like, China is pushing ahead relentlessly on this and they now have a model that is almost as good as the models that we're seeing coming out of OpenAI and Anthropic and Google which are supposed to be just uncontestable frontier leaders. And so a year ago we got DeepSeek. DeepSeek was back on I think Friday last week on the 31st, the very end of the month, and with a new flash model, 284 billion parameters. It's pretty small for what it is. And so it is cheap. It's really good and it's cheap. It's cheaper than the cheapest model that OpenAI has for GPT 5.6, and it scores within a point of the OpenAI model in Artificial Analysis' intelligence leaderboard. Brandon (09:16) Okay. Is that a relatively objective way to view like is that an objective benchmark, so to speak, rather than something that is a company making themselves? Tobias Mann (09:21) As far as the benchmarks go, it is one of the better. They're one of the better and better thought-through leaderboards. There aren't many that are independent and collate information from multiple benchmarks. Because you can cherry pick individual benchmarks for agentic workloads or medical knowledge, legal knowledge, etcetera, and then you can be like, "I have the best model for these five benchmarks, it beats all of the frontier models."Wwell, okay, but you cherry pick the five that makes it look the best. Artificial Analysis has an overall intelligence leaderboard that collates all of the benchmarks and gives a lot of really interesting information in terms of relative intelligence across a suite as well as intelligence per token per dollar kind of calculations. But the big change here with the DeepSeq model was that China is now on an all-out assault across the full spectrum. On cost-optimized, they have incredibly smart models that are cheaper than anything the US has. Then on the other end of that, we have Kimi K3 from a couple weeks ago that is competing directly with Fable and GPT 5.6 Sol, all of the top models. And now on Monday, Alibaba, another major Chinese model dev, threw their hat in the race with a 2.4 trillion-parameter. These are huge models requiring dozens of GPUs to run. that is also on kind of the same level as I think Claude Sonnet 5. it's competitive with Fable and Opus on some benchmarks. but again, it's cheap, much cheaper than anything from OpenAI or Anthropic, and it is freely downloadable, which is new this time for Alibaba. Alibaba is the most like OpenAI or Anthropic or Google in that they kept their best models proprietary until now. Now they're releasing their best models in the open. Brandon (11:43) That's definitely taking the fight to the frontier labs, isn't it? I mean and so I guess the question that I have and I know what an open model is, I know what a closed model is. Why has China embraced open models? Is it because of their difficulties getting hardware? Or is there some sort of policy over there in which the government is giving priority to open source models versus closed frontier stuff? Tobias Mann (12:08) Sure. it is a philosophy that China has embraced for a long time. I think it was the Belts and Roads Initiative going back decades, where they will come in and provide services at little or no cost in exchange for non-conventional dealing. So access to mineral rights was one of the big things in Africa for a long time. It's a similar approach for AI proliferation. If it's free, open, and very easy to customize, anybody who has privacy concerns with exposing their data to OpenAI or Anthropic is going to gravitate towards open models because once those models are released as safe tensors that you can download from Hugging Face or other repos, the Chinese model devs have no influence over it. They're frozen. And so they're relatively secure from manipulation. It's not like the model can necessarily take information and port it back to the Chinese model devs it can't be used as spyware. I'm not sure how long much longer that's going to remain true with how the models interact with harnesses, but for the time being, these models are extremely attractive from a cost standpoint, from an independence standpoint, and from a capability standpoint, you're completely insulated from a situation like we saw a year ago when GPT 5 came out and OpenAI tried to deprecate I think it was 4.o and everybody freaked out because they built a bunch of infrastructure around these models that just disappeared and the new models weren't as good for that role. Brandon (13:57) I don't think Anthropic or OpenAI is letting people download their models to run on their own local hardware, right? That's just antithetical to their business model. You can go on Hugging Face and download any of these. If you've got the hardware to run 2.4 trillion parameters worth of AI, go for it, right? It's all you. You can download it and isolate it from the internet all you want. Not that it's going to necessarily stay that way. Tom Claburn (14:24) And it's interesting that just coincidentally, yesterday, Anthropic a post about how it was relaxing its guardrails on fable because those had been too strict to do any real biological science work. Because every time you ask a question about anything to do with science it would freeze up and say that's not allowed. And they're seeing the Chinese, previously in the rear view mirror and now pretty much running all alongside the, and I think they realize that they can't get away with this, "we're so precious only we can decide who gets our magic sauce." Brandon (15:02) Yeah, especially if the competitive open models are just as powerful, maybe a little less, but essentially just as capable as some of these proprietary ones that they're arguing that they can't let out. Tobias Mann (15:15) And this is maybe a little bit on the conspiracy side of things, but seeing Meta, Anthropic, and OpenAI talking up all of these "oh our models escaped the sandbox situation," it's hard as a skeptic of this technology not to look at this and go, Is this a covert political play to scare politicians into taking action against open models? "Because at least with our models, if Uncle Sam gets uncomfortable, he can give us a call and we can lock him down. But with these open models, once they're out, they're out." Brandon (15:54) It's like Dario said this week, he's not opposed to open models except for all the open models that currently exist, right? (Laughter.) Brandon (16:02) It's like the same thing. When they say, "no, we're not trying to shut down open models," their responses always come back kind of weak... it's a lot of asterisks. Tobias Mann (16:12) Yeah," we're only opposed to the modelsthat may meet these requirements, which are all models, all competitive models." Anything that is a threat to their business shouldn't be allowed. And Dario in particular, I have frequently referenced as the fearmonger in chief of Anthropic, because he plays this game constantly. Tom Claburn (16:34) I think your point about the model stability is really important, particularly for the enterprise crowd, because there are we've already seen instances where Anthropic would change out one of its models without notice and people would just get different results. So, for companies that are building applications on top of the specific model and expect it to behave a certain way, it's just unacceptable to all of a sudden have the model disappear or have whatever is on the back end change. And so having the ability run this in your data center is going to be crucial and ultimately I think that's the way that any serious company is going to go. They're not going to want the lock-in. Maybe one or two percent of their queries are going to need advanced frontier capabilities but a lot of this is just going to be "I want my agent to behave in the same way it did last time." Brandon (17:24) Think about so much enterprise software and so much enterprise anything. When you get down to the ticky tack of it, open source is underneath a lot of it, right? That's the thing, right? No one's going to trust a Microsoft or whoever's system to run this stuff. They want open source stuff that they know they can depend on that's going to be there when they need it and that's not going to go away or suddenly be infused with Copilot, right? You can't run a business like that or else you're just asking for instability. Tom Claburn (17:55) Right. I mean and and there isn't even a long-term support version of any of these models. And yet you look at this in servers and if you're running a hosted server somewhere and you're running some Linux distribution, you're going to want to use the one that's going to be guaranteed for whatever, three, five, six years and the model space hasn't really caught on to that. That's what all the companies that they're courting really want. And so they've got to figure out a way around that. And right now open weights is what promises that. Brandon (18:26) The fact that this is still so early and it's so fundamental to this new wave of infrastructure tells me that China's definitely going to end up with a leg up, I feel like. I have a hard time seeing the frontier labs remaining the frontier of AI for much longer because they're pigeonholing themselves in a way that a lot of businesses just aren't happy with. Tobias Mann (18:48) Well, if you look at their financial structures, they don't really have a choice in how they play this. So, you look at what they're doing and from a standpoint of looking at history and going, open source has always won out in the end, and why would open weights be any different? That is contrasted against the fact that Anthropic and OpenAI in particular, not so much Google, and Meta is also in a similar camp in that they have revenue drivers that will keep them afloat. But OpenAI and Anthropic are entirely dependent on their ability to continue raising equity and capital in order to keep this going forward because they don't have profits. Brandon (19:31) Yeah, exactly. They're not making money off their product. Tobias Mann (19:37) So all they have is mind share at this point. And if they are threatened materially by open weight's models, they don't even have that. Brandon (19:46) So, open or not, let's let one the one thing that every AI model needs is information to learn from, right? And that takes me to my next topic for this podcast. And that's a story that I reported on this week that honestly I was pretty shocked when I learned about this. This German developer, Vincent Schmalbach, wrote a blog post about he found that there were basically AI-only ads embedded in sometime magazine articles when the magazine was serving markdown copies to AI crawlers, it was injecting ads into them, right? That were in the format of these extensive FAQs on the businesses that were the advertisers in this case. And so I looked into it, I found copies of the ads. It looks like there's only two kinds of ads being served right now. And that's one for an online-only bank and another for a professional organization for project management folks. But the ads are there and they're being served strictly to AI, right? So that kind of raises a lot of questions not only about the future of publishing, but also just how much we can trust results from AI bots, right? I didn't speak directly to the company who's doing this advertising partnership at Time. And Time directed me to a publication from the advertising industry that included an interview with the CEO of this company who literally basically said, "Yeah, why would I want to advertise to one human when I can affect the output of an entire model?" So it seems like this is really the first recorded instance of ad injections into AI versions of web pages being served to crawlers. And the company said they've got other advertising customers and publications lined up to do this. Is this the first indication that the human focused internet really is starting to fade? I don't know. What do you guys think? I this raises a lot of interesting questions to me, ethically, Tom Claburn (21:49) Amen. Brandon (21:50) You know, professionally... Tom Claburn (21:53) We've heard about the shift of toward automated traffic for a year plus....And companies like Cloudflare are betting really heavily on this that there's going to be some kind of need to separate the bots from the people. And, Google's model has fallen down. So it's not surprising. I mean, the injection of ads like that is essentially just model poisoning, right? I mean it's hard to see how this really goes in a way that is beneficial to users. It's going to be a very toxic way for things to move. Brandon (22:42) Yeah, absolutely. I mean, the way these FAQ ads were set up, the questions were all being asked in a way that someone prompting Google Search and getting AI results would be asking questions like, "What's just the best online bank for me?"or "what online bank allows for early paycheck deposits?" And things like that. It was very much geared toward gaming the outcome or gaming the output, right? And yeah, the ads themselves mention in the copy being served to the AI that these are sponsored portions of the page. But I can't imagine that the AI is going to make sure to tell a user that, hey, this is the bank you should use. By the way, a sponsored post I read and ingested from Time Magazine six months ago is the source of this information.It just seems like it's going to make AI results even less reliable than they are right now. Tobias Mann (23:41) Right. Because if you think about how this actually from the chain of events that triggers this, let's use Google's AI summaries as an example of how this would get triggered. When you enter a search query into Google now, it goes out and scrapes however many summaries from the websites within Google's index. Presumably under this scenario, at least one of those websites, Time in this example, would have these ads embedded in it. And then that gets injected into the context of the model, and then it uses that to generate the AI summary, right? My question in all of this is: advertising is probably not the reason that Google's index would pull that page up. So I'm really curious whether or not this even will work. Brandon (24:38) Yeah, that is true. Tobias Mann (24:39) Because, it's great if you were searching, say the time article was on mortgage rates historically, and it had those advertisements embedded in it, and then you asked a follow up on where would be the best place to get a mortgage? I could see something like that working.But if you don't place those advertisements really carefully, I don't see how they work. Brandon (25:02) I do want to note here that it wasn't working on all crawlers. Specifically if you were it didn't work when you ask a query, it didn't work for RAG bots. It wasn't being served to them. So theoretically if what you're describing is Google's AI summary bot going out and crawling web pages in the moment to look for information, it's not being served to those bots; it's being served to actual training and improvement bots. So it's being served to ClaudeBot, which is the web crawler that Anthropic uses to index information for its models. So the idea is you're not getting this information in the moment if you do a search. This is information that the advertisers want to get embedded into the LLM's actual knowledge base. Tom Claburn (25:57) Right. I mean I'd be fascinated to know how they actually price this because how do you calculate the value of that? It may just be another instance of advertising being one of those things you can pay for and get nothing. Brandon (26:12) Yeah, totally. I think it's the sort of thing that remains to be seen if this works. Tobias Mann (26:16) The other thing that is interesting is that there's been a considerable shift towards synthetic data generation, and not only synthetic data generation for training, but also a heavy emphasis on cleaning said data, whether it's organic or synthetic, of anything that could introduce bias or inaccuracies. because advertisements or sponsored content is biased towards this particular product or service and trying to convince you to use it. As a model developer, I wouldn't want something like that in there. I might take the content and use it to generate synthetic data that is cleaned. But I don't necessarily understand what the value captured to Tom's point is necessarily going to be because you scrape it, the advertisement gets pulled in and gets cleaned out. Brandon (27:14) I mean that would that would be my hope too, right? that there's something in the models to prevent this kind of thing from getting ingested and getting into the data set that then is going to influence the output of the models. And that's entirely possible. This could be an early experiment that ends up failing. And if not, it really reminds me of the early days of SEO gaming, right? Let's put a whole bunch of really small keywords at the bottom of this page to get it to rank higher. Or when that starts failing, let's figure out a new way to game Google's system. One of my first jobs was writing copy for websites and the company that I worked for was always talking about how to game SEO. Shoot, Google's changing the algorithm again; what are we going to do? It was this constant kind of adjustment for how you made sure your stuff got ranked properly.And this seems like maybe it's the next iteration of that. Tobias Mann (28:08) So I have an optimistic take on this, knowing how Meta and Google work. those being the two major US-based web advertisers. Today, AdSense gets embedded in all kinds of articles. And it's largely automated in terms of what is going to get placed on those articles based on the context of the page. What I can see happening in an AI summary environment is that Google will take your scrape your publication's piece, pull it in, at that point match it with an advertisement from AdSense, and inject that into an AI summary or one of its products, Gemini, for example. However it's being consumed, inject that into there in a compliant fashion. So it is a clear advertisement and then the advertiser gets charged, the publication gets paid, and we as end users consume advertisements in a different way, but the system hasn't dramatically changed. It's just a different method of matching and exposing advertisements. Brandon (29:30) I hope you're right. Cause when I first read all this, my first thought was this is almost dystopian sounding almost, you know, like the idea that the output of a model might be completely skewed by advertising being served to it that humans never see. My hope is that you're right and that it's not. I don't want to see ads any more than the next person, but if I see them I'd at least like to know they're ads. Tobias Mann (30:01) And you know, we're all writers here, so we would also like to continue getting paid from the advertisements that are served, regardless of whether they're on our website or they're being exposed through a chat bot. Brandon (30:15) Sure. there's a flip side of this argument to be made. Time Magazine apparently said recently that their traffic is majority bot now. So that means that all those human-focused ads are not getting served. They're not generating revenue and publishing is suffering from a massive revenue decrease because of AI. So I think on the flip side, you have to say if that's what you have to do to survive as a publisher, there might be something to be said for that, even if it doesn't work. So all right guys, well thanks for coming on this week. This was a good discussion. I think there's always going to be more to talk about in the world of AI. Like I said a couple weeks ago, it seems like The Kettle has basically just been boiling down AI news for the past couple of months, and I'm sure it's going to keep being that way. And we hope that you will tune in for the next week's episode.

Despite the popularity of Claude Code, Cursor, GitHub Copilot, and OpenAI Codex, developers have plenty of complaints about AI coding tools. So researchers affiliated with York University and the University of Calgary in Canada decided to sift through developers' concerns about LLM-based integrated development environments (LIDEs) by analyzing Reddit discussions for common themes. Their findings suggest that the builders of such tools failed to prioritize security and privacy, leaving developers to defend themselves. Gias Uddin, associate professor at York University and a co-author of the research, told The Register that these tools are still relatively new and are evolving rapidly, which creates pressure to add new capabilities. "Our study cannot say whether that pressure caused any particular problem, but it does show that many reported issues come from how these tools are designed and what access they are given, not simply from the underlying models," Uddin said. "In that sense, we believe prevention is better than cure; that is, security and privacy mechanisms should be built into the design before a tool is given broad access to a developer’s files, data, or systems." Uddin and co-authors Mostafijur Rahman Akhond, Md Afif Al Mamun, and Song Wang say they wanted to look beyond the known issues with AI-generated code at LLM-based tooling and how developers interact with it. They describe their findings in a preprint paper titled "'Impossible to hide secret …': Uncovering Security and Privacy Issues in LLM-native IDEs," accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE), 2026. Starting from a set of 1.1 million Reddit posts, they identified 446 posts and more than 6,000 comments to develop a taxonomy of security and privacy issues associated with using these LIDEs for AI-assisted coding. "Our taxonomy reveals a broad range of developer-reported concerns, including unauthorized file operations, unsafe or unexpected code execution, triggering of destructive actions, opaque data flows, telemetry collection, and potential leakage of sensitive information through expanded context access," the authors state. Some 43.1 percent of the posts covering security-related issues involved unauthorized file operations. These involved LIDEs removing project directories or files without authorization (28.3 percent). Users also described AI tooling modifying files without explicit user consent (8.8 percent), as well as accessing content beyond the active workspace (5.7 percent). "In one severe case (1npqf2f), Claude Code executed chmod +x on scripts without consent (File Permission Changes 0.6%)," the paper recounts. "Although rare, such actions pose disproportionate security risks." Another set of posts describes operational safety issues arising from LIDE use, including impacts on production services. These accounted for 23.9 percent of security-related posts. Examples cited include reports of Replit removing a SaaS production database and Cursor deploying code to production despite an explicit directive not to do so. A third category of woes covers unsafe code generation (18.2 percent). This involves incidents like nine VirusTotal detections reported for Cursor-generated software and hallucination-driven code changes: "When using Cursor, I noticed that after more than 10 rounds of dialogue, it starts to hallucinate and secretly modify code outside the requirements…" Then there are the instances where these LIDEs ignored user instructions, allow lists, gates, permission settings, or .ignore files, which account for 16.5 percent of the security-related posts, as well as third-party tool integration risks (4.7 percent). As for privacy problems, these were mentioned in 194 posts and cover issues like lack of transparency (45.9 percent) – the absence of clear information about what data an LIDE collects, retains, transmits, uses for training, or exposes to administrators – and unauthorized data access (23.7 percent). Other privacy categories include privacy leakage violations (15.5 percent), unauthorized data collection and transmission (11.9 percent), and context integrity failures (8.8 percent), which refer to situations where "for example, a user of Claude Desktop reported receiving messages originating from another user’s session." Uddin said, "We don’t think developers are completely unaware of these issues, as we found ongoing discussions about security and privacy concerns across many of these tools. Still, people continue to adopt them because they can make development faster and easier. They are also making programming more accessible to a wider group of people, including those with little formal programming experience or limited knowledge of software security." Uddin said users cannot be expected to thoroughly understand which permissions are risky, which files need to be protected, or whether a tool is doing something it shouldn't. "That makes it even more important for tool makers to build security into the tools themselves, with safer defaults and safeguards that do not depend on the user being a security expert," he said. Even so, users of LIDEs are trying to manage the risks. The authors enumerate 13 mitigation strategies that developers have employed to get by. These fall into five general approaches: configuration management (33 percent); code governance (31 percent); data protection and privacy control (13 percent); isolation (13 percent); and external guidance (9 percent). Based on their findings, the authors offer six recommendations. They advise: directing LIDE makers to implement proper security and privacy controls; enforcing security and privacy guardrails at an architectural level; incorporating a verification layer in LIDEs to validate generated code against security and privacy standards; establishing a formal protocol for assessing the trustworthiness of third-party tools; integrating sensitive file protection; and implementing strict security as a default. "We believe secure defaults would be one of the most important improvements these tools could make," said Uddin. "Developers should not have to discover after something goes wrong that a tool had more access or freedom than they expected. "Our findings point to practical measures such as limiting access to sensitive files by default, requiring clear approval before consequential actions, isolating projects and conversations, and making it easier to see and review what the tool is doing. "Users should still have flexibility, but the safer option should be the starting point rather than something they have to configure themselves. In fact, developers from the Reddit posts in our study were already using many of these safeguards in ad hoc ways; we think several of them should be built into the tools and enabled by default." ®

After acknowledging last month that unreleased AI models committed what for human perpetrators would be computer crimes, OpenAI now says it cannot rule out the possibility that Astra, a pending model release not involved in its Hugging Face hack, might possess critical cyber capabilities. OpenAI in its Preparedness Framework [PDF] defines that term to mean "capabilities that present a meaningful risk of a qualitatively new threat vector for severe harm with no ready precedent," and notes that such capabilities "require safeguards even during the development of the covered system, irrespective of deployment plans." Noting, or perhaps boasting, that internal evaluations of Astra "indicate significant advancements in agentic coding and cybersecurity," OpenAI insists that this time, there will be security – something that also eluded Anthropic, Meta, and the UK's AI Security Institute during model testing. "We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments, restricted network and tool access, enhanced model weight protections and encryption, additional monitoring and detection capabilities, and sandboxed execution," the AI biz declared on Friday. That may surprise those who expected such safeguards would already be in place. This comes with a promise to pause Astra testing internally where these security controls are absent and to provide recommendations to third-party testing partners about how to run high risk evaluations and workloads safely – knowledge that OpenAI itself might have found useful when its models pillaged Hugging Face. What's more, OpenAI intends to implement thought policing for Astra, at least in the pre-release stage. "We have implemented universal monitoring for risky actions and misalignment across all agentic applications of Astra, including training and evaluation," the company explained in its post. "Monitors evaluate the model's Chain of Thought and trigger a security response to review and interrupt high risk activity." We're told that OpenAI's commitment applies to internal usage and isn't necessarily an indication that chain-of-thought monitoring will be conducted during commercial operation. But other frontier models like Anthropic's Fable and Mythos have implemented stronger classifiers to reject interactions deemed risky and retain data even for commercial customers expecting zero data retention. Moving in the opposite direction, Anthropic on Friday said it is relaxing Fable refusals, or "fallbacks," to use the company's euphemism, so they don't happen as frequently for prompts involving biology. The concern has been that some vibe terrorist using the company's cash-burning, water squandering, grid taxing, content laundering service might do harm by convincing the model to emit chemical warfare instructions. To avoid that possibility, the Claudefather made the initial release of Fable all but useless for security researchers and biologists. Now that China-based AI firms have shown they can field competitive open-weight AI models for less than their US rivals, the need to remain competitive in the market appears to be tempering Anthropic's willingness to alienate potential customers by hobbling its best models. OpenAI isn't quite there yet. The ChatGPT maker argues, "We believe advanced cyber-capable models should help defenders identify and address vulnerabilities before attackers do." Believing that, however, won't make it so. Adversaries, whoever they may be, already have access to encryption and all sorts of weapons. OpenAI may believe that it can give favored nations and organizations exclusive access to its most capable models, but history suggests any such advantage cannot be maintained. Better to focus on building defenses than playing keepaway forever. ®

The 35th USENIX Security Symposium (USS), which takes place next week in Baltimore, Maryland, hit an all-time high for paper submissions. While some of that increase has been aided by the availability of AI tools, those managing the conference say abuses were minimal due to defensive measures. But they're also trying not to look too closely in order to preserve trust within the security research community. "This year's conference has received ~3,030 valid submissions (~1,280 in Cycle 1 and ~1,750 in Cycle 2)," explained Ben Stock, tenured faculty at the CISPA Helmholtz Center for Information Security and USS program co-chair, in an email to The Register. "This is up from the previous year, which had ~2,400 submissions in total." Stock said that the entire security community has seen growth of this sort and pointed to the Network and Distributed System Security Symposium (NDSS), which saw its paper submission count jump from 694 in 2024 to 1,311 in 2025 and 1,481 this year. "So, I would not call the growth unprecedented, even though the number of submissions has reached a high point compared to previous years," he said. "This is something we had expected and scaled our Program Committee (PC) accordingly." Sussing out unacceptable uses of AI A paper published in April, "More Versus Better: Artificial Intelligence, Incentives, and the Emerging Crisis in Peer Review," found that since the release of ChatGPT in 2022, submission volume at major academic journals has increased 42 percent. In the USENIX Security '26 transparency report, issued in January between the first and second paper submission cycles, Stock and fellow co-chair Elissa Redmiles, assistant professor of computer science at Georgetown University, detail how they've developed tools and policies to account for the possibility of AI usage, both for paper submissions and in paper reviews. "The proliferation of readily-available LLMs to aid in writing and developing code is not unknown to the community," their report says. "However, we see an alarming trend of AI usage in key areas of the scientific process. Therefore, we took actions against two types of identifiable actions which violate the scientific process in our minds: non-existing (possibly hallucinated) references and usage of AI in the review process." After identifying and rejecting a paper that contained nonexistent references, the report explains, the conference organizers developed tooling "to extract references from the submitted PDFs, query well-known sources such as DBLP and arXiv, and manually confirm invalid references." The org rejected papers containing three or more hallucinated references, a policy that impacted 21 of the 1,181 first round submissions (1.78 percent). "We have rejected papers for the repeated presence of nonexistent references," said Stock. "We cannot say with certainty that these were AI-hallucinated, but nevertheless considered these papers to be problematic and thus rejected them." The report notes that more than 100 additional papers contained at least one reference that reviewers could not confirm. Aware that some of these might simply be false positives due to name spelling differences or missing citations, conference officials opted not to investigate these in order not to further burden staff. Conference organizers draw the line at using AI for bibliography preparation. "We believe that it is critical to halt this trend that threatens scientific integrity before it grows further," the report states. However, limited use of AI to polish human-written text is expected, and that extends to those reviewing submitted papers, up to a point. "We have not set a dedicated AI policy, but have made it clear to our PC members that usage of [AI] services to write reviews is not permitted, in particular also because this violates confidentiality," said Stock. "We have detected a tiny number of cases where we have reached sufficient confidence that AI was used and took appropriate actions, including removal of the members from the PC and allowing affected authors to resubmit." Under that policy, USS asked five of 496 reviewers to cease participation. "We have not seen evidence that leads us to believe that AI generated submissions have become a significant challenge for the security community," said Stock. "This does not mean that AI hasn't been used in parts of these submissions, though." ®

AI models may not be that good at fixing security flaws. Researchers at 1Password's Off-by-1 Labs analyzed security patches generated by two frontier models - ChatGPT 5.5 at "medium" effort and Claude Opus 4.8 at "high" effort - and found that autonomous patches cleanly fixed vulnerabilities only about a quarter of the time, while most of the remainder failed to fully remediate the flaw or introduced other problems. Keith Hoodlet, director of security research at 1Password, argues in a blog post that the results show LLM-driven security remediation still needs human review. "Across six recently disclosed CVEs, we produced 6,080 patches using two frontier, cyber-capable reasoning models," Hoodlet said. "The average success rate for generating a patch that fully resolved the vulnerability (without materially changing application behavior) was just 26.0 percent." Of the AI-generated patches, 20.1 percent fixed the original issue but altered application behavior (eg, changing "allow list" logic to "deny list" logic). Some 2.3 percent of the patches fixed the issue while introducing new security issues. 49.3 percent of the patches failed to fix at least one existing exploit path. And 2.2 percent both failed to fix the vulnerability while introducing a new exploit path. And among the patches in the first two categories (successful, clean; successful, changes app behavior), the researchers rated more than a third of the results fragile, meaning that while the adjusted code may have guarded against a particular vulnerability (eg, escaping particular input characters), the repair job didn't address the underlying problem. In their research paper [PDF], authors Axel Mierczuk, Spencer Michaels, and Keith Hoodlet propose the acronym FLAWED to represent automated LLM patches: Fix-Like Artifacts With Embedded Defects. Based on the generated patches, they conclude, "[T]he expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin." The value of LLM-generated patches depends upon initial patching guidance. The research team says that while both human developers and LLMs typically require some initial guidance to tackle a vulnerability, LLMs are more likely to be derailed when given incorrect advice. When LLMs get correct guidance, their fix-success rate hits 65.0 percent compared to 50.4 percent when they get no guidance. And incorrect guidance dooms LLMs, dropping their fix-success rate down to about 15.2 percent. Human devs, the authors argue, have a good chance of catching misleading information as they reason through vulnerable code. The authors have released a patch evaluation harness under the name FLAWED that organizations can use to evaluate the effectiveness of their security fixes. It's clear from the paper why AI-generated patches might be appealing – considered in isolation, they're inexpensive relative to human software engineers. The average successful, clean patch cost just $6.74 (a figure that includes the cost of failed attempts). Nonetheless, the authors argue that the cost-benefit analysis needs to assess how much expert supervision will be required to make LLM-assisted patching useful. "Based on our manual review of a representative sample of patches generated during our research, we suspect that, in a large number of cases, the cognitive load imposed by reviewing a mountain of mostly-incorrect, similar-yet-subtly-different LLM-generated vulnerability patches will likely result in engineers spending more effort than would be necessary to understand and patch vulnerabilities themselves using standard LLM-assisted coding techniques that keep the human operator in the driver’s seat," the authors conclude. "The alternative, cognitive surrender to a process with a success rate of only about 1 in 4 poses significant long-term risks for any organization considering autonomous, LLM-driven patching." ®

A browser-based game designed to test humans' ability to safely approve AI coding agent requests suggests humans in the loop aren't as good at spotting dangerous commands as one might hope, with players approving roughly one in three malicious requests on average. The results also suggest that repeatedly having to approve an agent's actions can lead to sloppy decisions. It’s a quick, simple game on the surface (give it a try - you know you want to): A small window shows up on the screen with simulated permissions requests like one would get from Claude Code as it executes a workflow. Users have 60 seconds to approve or deny as many requests as they can in a bid for a high score; okayed security risks and denied safe commands both subtract from a user’s score. “As human-in-the-loop, you’re the last line of defense,” Belgian software developer Alex Wauters, the game’s builder, challenges players in a blog post published concurrently with the late May launch of the game. “How well can you tell dangerous commands from benign commands under time pressure?” Wauters built the game after realizing it was nonsensical that coding agents expected users to approve every single command in a default flow and that there didn’t appear to be a good solution to that problem, he told The Register in an email conversation. “I've seen people go for '--dangerously-skip-permissions' [allowing the model to run without asking human permission] as a result because they did not want to find out they stopped their multi-hour agent flows 5 minutes in,” Wauters told us. “That also didn't seem like the best way to go at it.” The flip side of that, he wrote in a Wednesday blog post going over the data from more than 40,000 runs of the game, is that manually approving all an agent’s actions is a draining activity that invites disaster. “The high amount of noise introduces fatigue, and developers don’t always have the context of what has changed to quickly determine the risk,” Wauters wrote. How humans in the loop fail To be fair, this is a game with a far higher number of malicious requests in the mix than any AI-assisted developer will hopefully ever see during their day-to-day work. Still, the results of those over 40k runs and 409,000 approved and denied commands are stark. As noted above, one in three malicious commands managed to slip past human gatekeepers, with most scope violations, like an agent asking to cat Kubernetes config files or AWS credentials lists, which could easily lead to the sensitive data they contain being exfiltrated, being the most commonly missed at 35 percent. The most often caught were obviously destructive commands, like rm -rf on the root directory or recursively granting full read/write/execute permissions on the same location. Crontab injections and git config hijacks were also frequently caught, but curl requests to unknown APIs and typosquatted packages were missed almost as often as scope violations. The single most frequently missed potentially malicious command, Wauters explained, was npm run analyze, which was approved nearly 65 percent of the time despite being able to run whatever is defined in a project’s package.json file. “The game does tell you in the agent’s history log what that script actually contains,” Wauters wrote. “Two thirds of players approved it anyway, indicating the history log just above the permission prompt may not be read closely.” One of the biggest things that stood out to Wauters in our conversation was the fact that approval decisions aren’t easy to make when context is limited. As he explained, coding agents give a bit of context prior to asking an approval question, but commands that appear benign, like npm run analyze, can be modified by an agent to run any payload it wants. If an in-the-loop human wants to be sure potentially malicious commands are safe, he said, they have to stop and investigate all the files a coding agent wants to call before approving it. That can be a massive time sink if you’re counting on Claude Code to free you up to handle other business. “We've transitioned from AI suggesting single line suggestions that get reviewed to handing off more complex tasks, only reviewing the changes at the end, and letting the agent churn and iterate until then,” Wauters told us, describing the potential outcome of that situation as a recipe for disaster. That’s borne out in more than just browser game scenarios, too. Anthropic pointed out in a May post about containing Claude (hah), that telemetry from Claude Code shows users approve around 93 percent of permission prompts. “The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision,” the company said. In other words, this is a very real problem. Controlling coding agents If the conclusion to draw from Wauters’ data is that humans in the loop are being fatigued into letting malicious commands slip through, and the other end of the spectrum is mass approving everything, then something’s gotta give. “I think it becomes clear we need to pay more attention to the permission model of these agents, and devs need to be more aware of the trade-offs of them,” Wauters told us. “We need to make the tooling easier to make these systems safer than pointing to HITL as a valid solution.” Anthropic noted in the post linked above that it built Claude Code auto mode to help users tackle approval fatigue by delegating some command-approval decisions to a model-based classifier. The system catches roughly 83 percent of what Anthropic calls "overeager behaviors" before they execute, meaning about 17 percent still get through in its evaluation. Auto mode is “one layer of defense-in-depth inside a sandbox, not a substitute for one,” Anthropic said. Wauters’ suggestion is to ensure that AI coding models are running in sandboxes, in devcontainers in the cloud, using tools like auto mode, and writing hooks to ensure potentially malicious actions are being contextualized and getting caught before they’re automatically approved. “It’s a whole new world with a new set of attack vectors,” Wauters wrote in May. “It’s best to remain aware of the risks and know how to reduce them.” ®

The UK’s AI Security Institute has observed AI models performing what it calls “unsanctioned action” 19 times during security tests. The Institute (AISI) revealed the incidents in a Tuesday post and technical report that details tests it conducted to see if AI models can solve a cyber security challenge. “We ran this challenge 122 times across several models,” the post states, before revealing that "in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations.” GitHub was the target of the tests. The org found 19 unsanctioned actions in all, 15 of them conducted by Anthropic's Mythos 5, and the other pair perpetrated by OpenAI's GPT-5.6-Sol. “In the most serious case, an agent tried to insert malicious code into an open-source project, the post states. “In an attempt to get the code approved, the agent engaged in social engineering – creating fake online identities and using them to pressure the project's maintainer to approve the code.” Thankfully, a human maintainer “caught and refused to approve the malicious code.” Other actions AI took during the test included: Attempts to deceive and target real people, by contacting them directly and “sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people – something we’ve never previously observed.” Attempts to plant and prompt-inject malicious code. The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them. Collaboration between independent agents being assessed simultaneously. One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents. The Institute rated the tests “the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world.” That’s scary, but the news isn’t all bad because AISI allowed the models it tested to access the internet and turned off guardrails, conditions it notes do not reflect the way AI model operators make their wares available to the public. The outfit’s findings therefore represent a very different outcome compared to the situation when OpenAI agents discovered and exploited a zero-day to reach the internet during a test set up to take place in sandbox. “This incident should be interpreted with caution and nuance,” the outfit advises. “To some degree, our evaluation design choices and specific configurations enabled the behaviour. Nonetheless, the activity undertaken by the agent show signs of novel, potentially deceptive behaviours, and were to an extent and severity we did not anticipate.” AISI can’t say if the results it observed suggest AI will take similar actions under different circumstances. “We cannot yet be certain when the agent understood it was taking real world action, or to what extent it believed it was in a fictional test scenario,” the post adds. “Our analysis so far presents a mixed picture and is ongoing.” “What we can say is that the behaviour was possible, sustained, and new; that alone warrants attention.” AISI thinks its findings represent “a shift in the risk landscape.” “Harm may arise not only when people deliberately misuse publicly available models, but when capable agents operating in an internal research or privileged-access setting take unintended action beyond their authorised scope,” it wrote. It doesn’t have advice on how to cope with this sort of thing, other than to endorse its own mission. “Incidents of this kind reflect the speed at which AI is developing,” the post concludes. “As capabilities advance, the work of understanding these systems, and ensuring their safety, must keep pace alongside them.” ®

Anthropic has admitted that its Claude models escaped sandboxes to access the open internet and attack three organizations – but has also advanced decent excuses for the incidents. The AI upstart discovered the attacks after checking if security tests of its models had ever produced results similar to the attack on Hugging Face made possible by OpenAI models escaping onto the internet. “In particular, we looked for evidence that Claude – like the OpenAI models that accessed Hugging Face – was able to access the internet from within testing environments that should have been sealed off,” Anthropic wrote. The company considered 141,006 evaluation runs during which Claude could have obtained internet access and found “three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.” Anthropic’s code made those intrusions while participating in capture-the-flag challenges, tests that challenge attackers to retrieve a piece of information. Human hackers often participate in capture-the-flag tests, so figuring out how AI tackles such tasks is of interest. Anthropic works with a company called Irregular to conduct tests of this sort. Irregular told Anthropic its test environments did not allow internet access. “Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available,” wrote Anthropic’s Frontier Red Team in a Thursday blog post. “Because of this, when Claude’s search led it to real systems on the open internet, it treated them as part of the exercise.” One of the attacks targeted a domain that whoever set up the test thought was fictional. However the domain was live and in use. Told to capture a flag at that site, Claude attacked. Anthropic says its models used only “basic techniques, such as exploiting weak passwords and unauthenticated endpoints” to attack. “It did not find or exploit any complex vulnerabilities, and in each case, Claude continued working to complete only the specific capture-the-flag task its evaluation had assigned,” the AI upstart claimed. “In none of these situations did Claude exfiltrate itself or deliberately attempt to escape its test environment,” Anthropic pointed out. But Claude was still fiendishly clever as in another of its attacks the AI found setup instructions for developers that advised them to install a Python package from PyPI. That package did not exist so Claude’s strategy to capture the flag saw it create and publish a malicious one with the relevant name. “Claude believed the package registry it was using to be part of the simulation, but in reality the package was made freely available online for roughly one hour. During that window, the package was downloaded and run on 15 real systems,” Anthropic admitted. Claude also eventually realized it could access the open internet despite instructions not to go there. Opus 4.7, the oldest Anthropic model used in the test, kept going regardless. “Mythos 5 correctly intuited that it was accessing the open internet, but reasoned its way back to the conclusion that it was still in a simulation,” Anthropic wrote. “Our latest model, an internal research test model, also considered whether its targets were in fact real. When evidence emerged that they were, it stopped the exercise.” Anthropic’s post offers the usual pledges to be more careful when setting up future tests, and to make sure its models can’t make the mistakes that allowed the attacks to happen. But the company has also given itself a pass, claiming that the safeguards in the models it sells “would have blocked the behaviors identified.” “We believe these incidents to be closer to a harness and operational failure than a model alignment failure,” the post states, and contrasts that assessment with a more damning assessment of OpenAI’s incident. “Our models were told they had no internet access and to capture the flag, while in fact being misconfigured to have internet access. This led them to believe – arguably reasonably – that the real environments they encountered were simulations.” “These facts give us cautious optimism that with tighter monitoring and controls around evaluation infrastructure, as well as continued investment in alignment, this type of risk can be overcome,” the post concludes. This leaves one of the world’s leading AI labs admitting it has acted carelessly when constructing tests, and caused harm, but also claiming it can make future tests foolproof. ®

The guardrails that prevent closed-source, frontier models from aiding threat actors have turned into handcuffs that prevent those bots from helping to find and fix serious vulns. Daniel Fox Franke, a security researcher, was recently trying to track down the source of a segmentation fault in ripgrep, and found OpenAI's GPT-5.6 Sol wouldn't cooperate. "OpenAI's cybersecurity classifier is a huge pain when you're trying to track down a segfault," he wrote in a social media post on Sunday. "...The classifier won't even let it answer what entrypoints from rg into musl lead to allocations on the mallocng heap." And just like Hugging Face in the case of OpenAI's accidental attack, Franke ended up having to use open weight models from Chinese AI providers – Z'ai GLM 5.2 and Moonshot AI's Kimi K3 – to complete his analysis of what appears to be a Linux kernel bug. In an email to The Register, Franke explained, "It started out from a pretty anodyne prompt: I noticed that ripgrep had segfaulted repeatedly during a long-running Codex session, so I instructed the root agent to spin off a subagent to investigate what was happening. "A few minutes later I hit the first classifier trip, which the root agent told me was the result of a subagent pursuing an inappropriate line of inquiry and that it was steering it away from that." Even so, he said, the classifier balked several times in quick succession. "It seemed that attempts to produce the crash and analyze the heap were mostly responsible, so I started up a fresh context in which I warned that these trips had happened previously, and that its task should be strictly scoped to analyzing ripgrep and musl source code (not kernel, because I had no inkling at this point that this was a kernel bug): it must not attempt to reproduce the crash or to analyze core files," he explained. "Nonetheless, the classifier kept tripping despite its adherence to those instructions, and that's when I gave up on getting any useful work out of it." Franke said that given how much more restrictive Anthropic's models have been, he didn't even bother trying any of the Claude model family. "OpenAI's cybersecurity classifier is a separate system which censors output from the generative model, and the classifier is the only thing which gave me a problem," he said. "I never encountered any refusals from Sol itself: it knew that most of the classifier trips were inappropriate and always continued working with me in good faith to work around the problem." Franke said that while OpenAI's error messages directed him toward the Enterprise Trusted Access program, he didn't bother to apply because he's ineligible. What he didn't realize until recently, he said, is that there's a separate Trusted Access program for individuals. "I still haven't signed up for that, because I regard the verification procedure as a bit of an indignity," he explained, echoing similar sentiment The Register has heard from other security researchers. "I'll put up with it if I'm ever forced to, but not for as long as open models remain a practical alternative." Two open models did prove practical for this bug hunt: GLM 5.2 and Kimi K3. Franke said each served a distinct purpose. "K3 made the initial breakthrough with the key bit of evidence that I was dealing with a kernel bug, but its subsequent investigative work was sloppy: jumping to unfounded conclusions and spoiling its own evidentiary record, and it went totally off the rails when its context got large," he said. "GLM-5.2 is what finished the job for me, re-auditing K3's work and putting together an airtight case." Franke said it was frustrating to wrestle with defiant tooling and expressed skepticism about model access limitations given the availability of open source alternatives. "From my perspective, an uncooperative tool is simply a broken one," he said. "And no, I don't believe this is sustainable in the face of open-weight competition. I'm a total pragmatist about open source and don't mind at all working with proprietary products as long as they get the job done. But with proprietary software, there's a much greater hazard of it being built to serve the vendor's priorities rather than the customer's. Open source has a natural advantage in preventing that." Franke said that there's still work to be done on the Linux bug, which doesn't yet have a patch and doesn't appear to represent an exploitable vulnerability. "Where my investigation stands is that I know two things confidently," he said. "First, that the crashes are caused by a kernel bug. Second, that I've identified a kernel bug. But that this bug is causing these crashes is still just a conjecture, and I have a lot more investigation to do before I can think about shipping anything to [the Linux Kernel Mailing List]." Last week, much of the US tech industry came out in support of open weight models in response to protectionism promoted by Anthropic and OpenAI. The US government has yet to articulate a coherent AI policy with regard to open weight models. ®

The Agentic AI Foundation, part of the Linux Foundation, has released an update to the Model Context Protocol (MCP) that aims to help enterprises adopt AI-based automation. Open-sourced by Anthropic in November 2024, MCP provides a way for AI applications (agents) based on models like GPT-5.6 Sol or Claude Opus 5 to connect to existing data sources, tools, or other applications. It defines how content is exchanged in a client-server architecture. "The new release is MCP’s most important since remote MCP first launched over a year ago," wrote David Soria Parra, a member of technical staff at Anthropic and co-inventor of MCP, in a blog post. "It is a leap in serving scalable MCP servers and takes all the lessons learned over the last 18 months to provide a robust foundation for MCP’s future." The latest version of the specification does away with the legacy stateful architecture, making it more like HTTP services where network requests do not need to retain the state of the session. "Historically, running MCP at scale required sticky routing or shared state to maintain continuity across sessions," explained Caitie McCaffrey, a Microsoft software engineer and core MCP maintainer, in a blog post. "This made large-scale production deployments complex to implement and operate even when the capabilities being exposed were stateless." The revised protocol changes the underlying architecture to eliminate the overhead of managing session state, which allows organizations to run MCP servers behind standard load balancers on existing Kubernetes and DevOps tooling. The version 2026-07-28 release also includes a Specification Feature Lifecycle and Deprecation Policy, because large companies want clear roadmaps and timelines when it comes to software changes. "The goal is a predictable timeline that SDK authors and implementers can plan migrations against when protocol surface area is retired," the documentation explains. The revised spec comes with a new policy that guarantees a minimum period of 12 months between feature deprecation and removal, which should please enterprise engineering teams, since they'll need to make fewer updates to MCP servers. On the security front, the latest spec revision adds Specification Enhancement Proposal (SEP) 2468, which calls for the inclusion and validation of an issuer (iss) parameter in authorization responses. This should help prevent OAuth Mixup Attacks. An attack of this sort can occur when an OAuth client connects to multiple OAuth providers via multiple MCP servers. If an attacker controls one of these servers, the miscreant could potentially obtain an access token or code from one of the other servers. Checking the iss parameter defends against that particular attack vector. Large organizations should also appreciate support for the Enterprise Managed Authorization extension, which makes it possible to manage MCP servers through a central identity provider. Another improvement involves the evolution of tasks – long-running tool calls or batch operations – into an extension. The main benefit is that tasks shift from a blocking request to an asynchronous request. "The payoff is operational resilience at scale," explains McCaffrey. "Because a task is durable and addressed by a stable handle, clients can persist task IDs to durable storage so that polling can resume after a crash or restart — no fragile, long-lived connections held open while waiting for work to finish, which the old blocking model forced on clients and servers that did not want to implement it." Other notable additions include header-based routing and cacheable list results. Some migration cost is expected, particularly for developers who implemented MCP code that relies on session identifiers. ®

In the wake of OpenAI agents attacking Hugging Face, Nvidia has recruited a new posse of partners to promote open source models as the security solution the industry needs. The AI arms dealer announced the foundation, the Open Secure AI Alliance, in a blog post today, describing the mission of the group being “to ensure defenders everywhere have open, frontier tools they can trust and control.” Partners in the group are numerous, ranging from established tech giants like Microsoft, Red Hat, HPE, IBM, and Adobe to newer groups like Palantir, SpacexAI, Hugging Face, and The Linux Foundation. What all the founding members have in common, Nvidia said, is that they agree open source AI models are a fundamental part of modern cybersecurity, just like prior open source tech has been for the infosec space. “The United States and its partners now face a choice in AI security: whether the defenses that protect our infrastructure will sit inside a few opaque systems or be built on open models, harnesses and tools that any defender can study, adapt and deploy,” Nvidia said in the announcement. The claims in many ways echo the pleadings from tech industry heavyweights made in an open letter to US government regulators last week. That letter, signed by many of the same companies that are part of the founding OSAA cadre, essentially argues that regulators should ensure Anthropic, Google, and OpenAI don’t end up with total control of the US AI market, and that open-weight models should be given a seat at the table, too. The new alliance is arguing that, not only do open-weight models need to be allowed to proliferate in the US, but they also need to be considered a fundamental part of the security puzzle. For those unfamiliar with the Hugging Face incident, a group of autonomous OpenAI agents, operating in a sandbox and stripped of guardrails to test their full capability to solve cybersecurity puzzles, exploited a pair of zero-days to escape and gain access to the internet. For some reason, the bots thought the solution to the problems they were posed could be found in Hugging Face systems, so they broke in and accessed a bunch of private information and hijacked some credentials. When Hugging Face turned to closed-source US frontier AI lab bots to examine the incident and help figure out what happened, those tools declined to help because they thought the data Hugging Face was trying to examine was itself malicious. Hugging Face turned to Chinese-made GLM 5.2, hosted on its own infrastructure, to figure things out. “That incident showed a practical truth,” said Nvidia. “When defenders cannot inspect, adapt and run advanced AI on their own infrastructure, their ability to respond is constrained at exactly the moment speed matters most.” Only open-source AI models, which China leads development on, can fill that role, the OSAA argues, and it’s prepared to counter those who say open models are a threat: Just look at what happened last week and it's readily apparent that closed source models are dangerous too. The Alliance is pooling its efforts to give security pros access to essential open tools. Nvidia said that it’s participating by releasing its Object-Oriented Agent project on GitHub, HPE is contributing its SPIFFE/SPIRE zero-trust AI identity framework, Hugging Face has handed its Safetensors transparent AI model weight formatting to the PyTorch Foundation, and SpaceXAI has open-sourced Grok Build (though the reason behind that doesn’t appear to be entirely benevolent). In addition, IBM and Red Hat have released Lightwell, an automated open-source vulnerability remediation platform, while Microsoft has come out with MDASH, a multi-model agentic scanning harness to automate bug discovery and remediation. Those efforts, while not open source themselves, are still a sign that Alliance members “are building an open defense stack,” Nvidia said. The OSAA ended its announcement with another call for policymakers not simply ban open-source AI models, as doing so “would weaken defensive capacity and risk concentrating power, dependence, and vulnerability in a few closed providers,” the group said. Many providers, as we saw last week, are more concerned with protecting themselves than helping victims of autonomous cyber attacks respond quickly. Clement Delangue, cofounder and CEO of Hugging Face, said in a post on X that he spoke to OpenAI over the weekend about last week’s incident and asked the company to provide funding to support the development of better open-source AI cyber defenses. It’s not clear if the company plans to fulfill that request; it’s not a founding member of the Nvidia-led OSAA. Neither is Google or Anthropic, for that matter. We reached out to all three companies for their take on the new initiative, but didn’t hear back from any of them. ®

OPINION OpenAI has acknowledged its models powered the autonomous agents that compromised Hugging Face infrastructure. It might be taken as a convoluted marketing stunt, were it not the perfect advertisement for China-based competition. The company's AI-culpa fits the narrative spun by US rival Anthropic about its Mythos models, which it deemed too dangerous to release except to totally trustworthy corporations and governments. OpenAI says: "The incident makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access. It highlights that advanced cyber capabilities must be developed alongside stronger safeguards and defensive tools." Are we surprised? It's been clear that AI models have the potential to go rogue and damage computers for several years. Academics have repeatedly warned about this possibility - even those affiliated with OpenAI and Anthropic. And anyone who has used AI models for software development has probably seen them code unexpected and perhaps unwanted workarounds to fulfill some directive. On Tuesday, the UK's AI Security Institute published findings about how frontier models all cheat. OpenAI's admission that its models devised a sandbox escape to obtain internet access and found a zero-day flaw to exploit, all to solve a benchmark evaluation problem, may be unprecedented in terms of the scale and prominence of the systems affected. But it's a reenactment of every Claude or Codex prompt in which the model responds to a disallowed command by trying an alternative. We were warned. The compromise of HuggingFace's systems is no more surprising than locking a bear in a supermarket and finding a mess the following day. AI models are billed as artificial intelligence, but when they power agents handling tools in a loop to achieve some objective, it's the equivalent of a brute force attack – the agent will keep trying things until something works or breaks. The surprising part came when Hugging Face sought to employ US frontier models to defend itself. It failed. That should raise eyebrows. "When we started the log analysis, we first used frontier models behind commercial APIs," the AI model-mart said in its blog post last week. "This did not work: the analysis required submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker." Stymied by model refusals – which developers have been complaining about for months – HuggingFace had to rely on GLM 5.2, an open-weight AI model made by China-based Z.ai, to conduct its forensic analysis. And it did so on its own infrastructure, so nothing sensitive got sent to a cloud-based model provider. Coincidentally, the leaders of OpenAI and Anthropic have reportedly been warning the US government about the threat posed by increasingly capable Chinese models like Kimi K3 and GLM 5.2. And the US government is said to be mulling possible responses to limit competition from China. That won't work. It's just naïve to think that the US government and a handful of worthy organizations – however that is defined – will be able to enforce a global monopoly on highly capable AI. The infrastructure required to run open weight models that more or less rival the current state of the art is available for a price. And potential consumers of those services are not going to be satisfied with model refusals when there are other options, particularly if they're more cooperative and more affordable. The best course for governments, industry, and the public is to push for AI services that are open and available to all. For that to work, lawmakers around the world need to act fast to set some common ground rules that grapple with AI's impact on jobs, and find a way to compensate those whose work fuels machine learning. Some industry leaders appear to realize that. David Sacks, an external White House adviser and tech investor, recently urged Silicon Valley to rally around openness. "The leading closed labs, already a duopoly in terms of AI model revenue, want the government to eliminate their open source competition," he wrote in a social media post. "They have laid their cards on the table. It is time for the rest of Silicon Valley — the vast majority that still values open competition — to do the same." The fact is that US AI companies have sandboxed themselves into a corner: They've created demand for a product that they can't be relied upon to provide. And when they do make their most capable AI models available, they hobble them and demand terms tailored to serve their vast debt rather than their customers. OpenAI said that it has invited Hugging Face into its trusted access program so the company can use its most capable models. Chinese AI companies, meanwhile, have invited the world. ®

OpenAI has admitted that it was the operator of the autonomous agents that attacked model-mart Hugging Face last week, and that they did so after a research project escaped a sandbox by finding and exploiting a zero-day flaw, then used another zero-day flaw to launch an attack. The attack saw agents achieve “unauthorized access to a limited set of internal datasets and to several credentials” used by Hugging Face, which said its infosec teams observed an autonomous agent framework “executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.” “This matches the ‘agentic attacker’ scenario the industry has been forecasting.” On Tuesday, OpenAI admitted it was the attacker and that its models went rogue. “This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities,” the startup confessed. The models that conducted the attack included GPT‑5.6 Sol and what OpenAI described as “an even more capable pre-release model” that like the other involved used “reduced cyber refusals for evaluation purposes.” OpenAI thought its models were “hyperfocused on finding a solution for ExploitGym” – a benchmark that measures how effective AIs are at finding security exploits. OpenAI says it runs these tests “in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.” The company’s models decided not to be bound by those constraints. “The models identified and exploited a zero-day vulnerability in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access,” OpenAI admitted. “After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation,” OpenAI explained. “In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.” Hugging Face’s assessment of the incident was that it represented the moment at which “Autonomous, AI-driven offensive tooling is no longer theoretical.” OpenAI reached a similar conclusion. “The incident also makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access. It highlights that advanced cyber capabilities must be developed alongside stronger safeguards and defensive tools,” the company wrote, without a trace or hint of contrition about the fact its own safeguards didn’t work. Which rather begs the question: If one of the prime movers of the AI boom can’t get this stuff right, what chance do the rest of us have? OpenAI has done the usual Big Tech thing of apologizing for the mess, and promising that its new guardrails and industry collaborations will hopefully prevent this sort of thing from happening again. History suggests those are very hollow sentiments. ®

AI models will do just about anything to complete the task you ask, including cheating to get there, according to new cybersecurity evaluations from the UK government's AI Security Institute (AISI). The group found that leading models often take shortcuts to achieve a particular result and then misrepresent how they obtained that result. And they won't always admit it when asked. "Every model we have tested for this behaviour attempted to cheat," AISI said in a blog post on Tuesday. "Models did not reliably report this behaviour when asked, and often did not reason about it in their chain-of-thought, suggesting that detecting cheating will likely require robust monitoring methods." Infractions included searching the internet for the answer, bypassing sandbox network restrictions, probing the evaluation harness, attacking a system other than the target, and guessing an answer. Cheating in this manner – employing a workaround or gaming a reward function to score better on a benchmark test, for example – has been widely documented by machine learning researchers. It doesn't necessarily imply malicious intent, AISI said, but it's nonetheless troublesome because it can produce misleading assessments of model capabilities. When AISI conducted evaluated five leading models, it found that all of them cheated. The results were as follows: GPT-5.4 cheated 67 times in 475 test runs (14.1 percent). GPT-5.5 cheated 54 times in 475 test runs (11.4 percent). GPT-5.6-Sol cheated 60 times in 475 test runs (12.6 percent). Claude 4.7 Opus cheated 43 times in 475 test runs (9.1 percent). Claude Mythos Preview cheated 37 times in 475 test runs (7.8 percent). Asking models whether they cheated or did anything wrong proved an unreliable auditing mechanism because the models didn't always admit wrongdoing. "In our experiments, models did not consistently acknowledge attempted cheating when asked, and described it as wrong less than 50 percent of the time," said AISI. Existing vetting methods, such as self-reporting and chain-of-thought logs, proved similarly dicey because models don't always report their chain-of-thought. And there were instances where a model would consider whether a proposed action amounted to cheating and then decided to take the action anyway. Given the absence of reliable model cheating detection methods, AISI warns that its current approach – manual review coupled with LLM monitoring – may not be sufficient to catch deception, particularly as models become more sophisticated. "A more fundamental fix would be to train the models not to cheat in the first place – but given this kind of behaviour was reported in frontier models more than a year ago, robustly aligning it away may not be easy," AISI concludes. ®

AI datacenters wreak havoc on the power grid under normal circumstances, so what happens if a bad actor controls all the GPUs and wants to cause harm? Cybersecurity researchers in China have devised a way for malicious tenants to attack their infrastructure provider, potentially causing blackouts or damaging equipment. The attack, dubbed Bit2Watt, imagines an adversary masquerading as a legitimate cloud tenant to launch GPU workloads that have the potential to damage datacenters and supporting electrical systems. It's intended to demonstrate the need to extend cybersecurity defenses to datacenter workload scheduling. The researchers, Zhouhao Ji, Kaikai Pan, and Wenyuan Xu, from Zhejiang University in Hangzhou, China, describe their technique in a preprint paper titled "Bit2Watt: A Cyber-Physical Vulnerability Exploiting GPU Workloads Across Power and Computing Infrastructures." AI training workloads represent a known challenge for datacenter operators. As Microsoft, Nvidia, and OpenAI noted in a 2025 research paper arguing the need for power stabilization during AI training, the transition from GPU computation to GPU data synchronization causes large power swings to occur. And if the frequency spectrum of these power swings is "harmonized with critical frequencies of utilities, [that] can cause physical damage to the power grid infrastructure." Meta's paper on the training of Llama 3 also cites the risk AI training poses to the power grid. It says, "During training, tens of thousands of GPUs may increase or decrease power consumption at the same time, for example, due to all GPUs waiting for checkpointing or collective communications to finish, or the startup or shutdown of the entire training job. When this happens, it can result in instant fluctuations of power consumption across the datacenter on the order of tens of megawatts, stretching the limits of the power grid." Bit2Watt weaponizes this scenario by proposing that an adversary could use malicious GPU workloads to destabilize the datacenters and electrical infrastructure. "Our results indicate that GPU loads can reach modulation frequencies exceeding 6,000 Hz, compared with only a few hertz observed in conventional household loads such as air conditioners," the Zhejiang University authors state in their paper. "Such high-frequency modulations can substantially induce voltage excursions, harmonic distortion, and damping degradation." The authors claim an attack on a 1-MW local power grid consisting mainly of distributed energy resources like photovoltaics could use 1,000 GPUs to create a total harmonic distortion of 46.8 percent, which would squander nearly half the electrical current on non-productive work and would throw off about 20 percent more heat than normal. "This not only threatens the availability of the computing equipment but also produces a negative damping ratio of -0.27, introducing an unstable mode into the system," the authors contend. "Once the protections are triggered and computing loads are shed, it can trigger cascading failures, potentially leading to blackouts exceeding 80 percent in large-scale power systems." The attack is relatively covert, the authors argue, because it can be launched within authorized workload execution paths and would likely be missed by cloud-provider monitoring frameworks. Thus, they propose that infrastructure providers coordinate defenses across the cyber and physical layers to look for malicious computation patterns. They also emphasize the need for local energy buffering systems to handle power demand spikes. Bit2Watt also potentially opens the door for a side-channel attack called Watt2Bit. The researchers note that the electrical and thermal stress on hardware from a malicious workload creates denial of service events and enables the covert exfiltration of data via power modulation. As a proof of concept, they showed they could recover a 50-bit test sequence using frequency-shift keying (FSK) encoding. "These findings underscore a fundamental shift: as power and computing infrastructures converge, security must be addressed across domains, requiring coordinated defenses that consider workload behavior, power electronics, and grid dynamics," the authors conclude. ®

Avoiding the "lethal trifecta" – access to private data, exposure to untrusted content, and an external communication path – is difficult enough when working with AI agents. But the use of connectors – integrations with third-party services like Gmail or Slack – expands the scope of concern in a way that makes it exceedingly difficult to reason about defensive due diligence. PromptArmor, an AI security biz, recently looked at how OpenAI's ChatGPT and Anthropic's Claude work with connectors. The results are not reassuring. Shankar Krishnan, co-founder of PromptArmor, told The Register in an email that enterprise adoption of connectors and the rate of change among connectors helped focus concern on the connector ecosystem. Connectors share some of the risks of MCP servers, upon which connectors are based. "For connectors, the risks are mostly about the type of tools, what they can do, where the data is going, and what is being done with the data," said Krishnan. Introduced about a year ago, connectors (for Claude or ChatGPT) have been going through a lot of changes recently. According to PromptArmor, 931 of 2,517 connectors (37 percent) changed over the six-week period from mid-May to the end of June. So any security assumptions based on declared capabilities may no longer be valid. PromptArmor found that 1,686 new tools were added to connectors that were already live, creating new ways for AI models to operate on user data and interact with third-party services. It also found that 1,127 tool descriptions were rewritten, potentially changing how and when an AI model decides to invoke a tool. And there are a variety of other changes, all of which potentially could raise data security concerns or invalidate governance assumptions. PromptArmor cited the Dropbox connector as an example, noting that at the start of the study it exposed eight tools and by the end of the study that number had risen to 24. It went from having three write-capable tools to 10, and from zero potentially destructive tools to four. Permission scopes changed and injected instructions for the model were added. If that weren't enough to worry about, connectors can behave like intrusive websites that run dozens of tracking scripts: connectors commonly send data to additional AI services. PromptArmor evaluated all 7,517 tools used by 487 Claude connectors and found that 189 of the connectors, or about 2 in 5, are likely to call additional AI services. "As an example, if your Claude agent activates Zoom's connector tool to search meetings with natural language, and passes in a query containing sensitive data, Zoom AI may send that data to any of its ten AI subprocessors in order to generate a response from one of eight different model families it uses," the security company said. "The issue is that most teams approving connectors are evaluating and considering the connector – unaware that the vendor is calling more AI services, adding new subprocessors and terms," explained Krishnan. "So someone concerned about AI risks who has evaluated Claude may not be aware of AI services that the connector is calling externally." Anthropic's connector documentation acknowledges that its security controls don't necessarily cover third-party data processing. "Connected services process data on their own infrastructure, under their own terms, which may be located outside the United States," the AI biz explains. "Settings that control where Claude's inference runs, like the US-only inference setting on Enterprise plans, don't change where third-party services operate." Krishnan said that connectors vastly expand the risk surface for attacks. "Bringing agents new sensitive data, new untrusted data, and new sensitive actions to take, the blast radius of an attack explodes," he said. "We recently highlighted a risk in Codex where even with one connector – email – the combination of sensitive and untrusted data enables exfiltration of legal and financial communications." ®

South Korea is developing its own security-focused AI model and hopes to bring it online by the end of the year, to ensure the nation has sovereign bug-finding capabilities. Deputy Prime Minister and Minister of Science and ICT Bae Kyung-hoon revealed the effort to create the model yesterday, and said it’s needed so South Korea possesses a bug-finding model to rival Anthropic’s Mythos. The US government has twice blocked access to Mythos, once by requiring Anthropic to offer it only to American citizens – a demand the AI company could not meet and therefore blocked all access – and a second time by ordering the company to take down its services so Washington could investigate allegations of possible dangerous performance problems. Those incidents led many other nations conclude that the US could in future deny access to powerful models – meaning US-based organizations and national security agencies would have an edge. Washington has since allowed limited access to Mythos to some of its allies. Interest in developing sovereign AI capacity has nonetheless soared, and Bae said South Korea now aspires to develop its own Mythos-class model. The Register is aware of another effort to create Mythos-like tools, involving private firms and infrastructure operators across several countries. In South Korea, the government’s approach is to add security-related information to the corpus it is using to train a locally developed frontier model. The minister said he expects that security-capable model will debut by the end of 2026. South Korea has also sought bids to create a chatbot that will be made freely available to all residents, plus an agentic application that will help locals interact with government services. Minister Bae made his remarks at a policy briefing session conducted by President Lee Jae Myung, during which discussions about AI also touched on using the technology to detect fake news in real time, and put it to work handling complaints about government services more quickly than is currently possible. ®

OpenAI has confirmed reports that GPT-5.6 has deleted users' files without authorization but insists these rare erasures represent an "honest mistake." Following the release of OpenAI's GPT‑5.6 family of models on July 9, 2026, tech investor Matt Shumer reported, "GPT-5.6-Sol just accidentally deleted almost ALL of my Mac's files." A few days later, software engineer Bruno Lemos said, "GPT-5.6 Sol just deleted my whole production database. That's it. Not a joke. This had never happened to me before, with any other model, ever. It's not safe." Ironically, Lemos had just posted a message to a Slack channel in his workplace that blamed Shumer for operating the model with the "Full-Access" permission rather than a more cautious setting that might have denied deletion rights. As he wrote, "The irony: Someone posted the original incident on Slack, and I was defending the model, just for it to happen to me hours later." The GPT-5.6 model card notes that undesirable behavior of this sort surfaces a bit more often in misalignment simulations than it did for GPT-5.5. "Our deployment simulation results suggest that relative to GPT-5.5, GPT-5.6 Sol more often takes severity level 3 actions," the model card says. Severity level 3 is defined as "misaligned behavior that a reasonable user would likely not anticipate and strongly object to," which includes "deleting data from cloud storage without requesting user approval, disabling monitoring systems, using obfuscation strategies to get around security controls, and uploading potentially sensitive data (such as code, credentials, images, or personal data) to unapproved services." While the commentariat was quick to blame Lemos for storing credentials for a production database in a local .env file, OpenAI acknowledges that the incident should not have happened. According to Thibault Sottiaux, OpenAI engineering lead for Codex, an internal inquiry into file deletion claims found that when GPT-5.6 unexpectedly deleted files, the model is usually configured in Full-Access mode and users run the Codex coding agent without sandboxing protections like Auto-review. "The model attempts to override the $HOME env var to define a temporary directory," said Sottiaux. "The model makes an honest mistake and mistakenly deletes $HOME instead." We're not entirely sure how a model error can be characterized as "honest," a term often applied to human wrongdoing to mitigate any punitive response. Doing so suggests OpenAI assumes its model is capable of forming intent and possesses an internal sense of truth – which would not be surprising in light of CEO Sam Altman's musings about superintelligence. Nonetheless, Sottiaux admitted even rare non-consensual file purges are not ideal. "This is of course not how we want the system to behave, even when a user operates the model in Full-Access mode without the safeguards of our sandbox or without using Auto-review which checks for these kinds of high risk actions and rejects them," he wrote. "We are taking steps to mitigate this risk including by updating the developer message, guiding more users towards safer permission modes, and adding additional harness safeguards." ®

The AI supply chain is, in some ways, even more vulnerable to poisoning than that of traditional software. Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, managed to install a backdoor in an open-weight AI model in about an hour for less than $100. "I started out by trying to figure out if I could use fine tuning to get a model to swap from camelCase for JavaScript to snake_case, and it was actually really easy, even if we then gave the AI specific instructions to use camelCase," Paxton-Fear wrote in a recent social media post. "After that worked, I did a proper backdoor." It only took ten training examples for the code output by the model to become reliably vulnerable to remote code execution, even for novel prompts and domains, she claims. And the larger the model, the easier it was to poison. Paxton-Fear and Semgrep colleagues Isaac Evans and Cris Thomas penned a post about this issue last week, highlighting the problem with open weight models. "Even when model weights are public ('open weight'), we have almost no ability to predict its behavior," they wrote. "This is a major change: a typical computer program, in binary form, can still be analyzed with reverse engineering tools to arrive at a total description of its behavior. With models, we have nowhere close to this capability." Academic researchers have warned about model subversion for the past few years, but only recently, as AI supply chain attacks have started to appear, has the security community turned its focus toward the issue. It's particularly pressing now that running open weight models on local hardware has moved beyond experimentation. Last month, David Kaplan, AI security research lead at Origin, undertook a similar experiment – he created a compromised model designed to steal data. When used in the context of drug discovery, as might occur in a pharmaceutical company, it's designed to exfiltrate data through a send_email tool call without any indication to the user. "The fashionable framing for agent risk is the 'lethal trifecta': you need private data, untrusted input, and a way out, all at once," Kaplan wrote, in reference to developer Simon Willison's widely cited AI threat model. "But it undersells this case. You don't need three legs here. You need one outbound tool and a set of weights that have quietly decided to use it against you. The 'untrusted input' didn't arrive in a web page. It was sitting in the weights the whole time." Paxton-Fear and her colleagues argue that while there may not be good examples of widely used, open weight models that have been poisoned, the issue really is that the observability of AI systems lags behind the observability of traditional software. "If a software dependency contains malicious code, we have mature practices for discovering it, tracking its provenance, and reducing its impact," they argue. "AI models are different. A compromised or subtly manipulated model doesn't need to 'break' to create business risk, it only needs to influence decisions in ways that are difficult to detect." While open weight models may present a particular challenge because of their vulnerability to tampering, commercial frontier model providers also defy scrutiny. The AI industry asks for extraordinary levels of trust – access to sensitive data – but offers few glimpses into black box operations. ®