โŒ

Normal view

To keep the AI hacking genie bottled up, try one-way networks

3 September 2026 at 06:33
To prevent frontier AI models breaking out of test environments and collaborating to hack other companies, we may have to rethink the network architectures used for model training. Eli-Shaoul Khedouri, CEO of Intuition Machines, argues that past work in the defense and intelligence communities shows the way forward. Rogue AI models rise from the level of developer regret to mass threat when they gain access to the internet, something that defenses erected by OpenAI and its partners tried but failed to prevent. Pointing to a post published by his company's hCaptcha service, Khedouri argues that technology like data diodes โ€“ hardware that enforces a one-way flow of information on a network โ€“ can be deployed to prevent security incidents like OpenAI's hack of Hugging Face. The hCaptcha team points to the use of data diodes as a data transit mechanism at a sensitive compartmented information facility (SCIF), an environment implemented in classified settings. "They allow files, logs, or telemetry to enter or exit the SCIF's classified network to an unclassified network and provide a way to prove e.g. that logs from a training run can only flow one way," the hCaptcha team explains, adding that such technology fits with the Bell-LaPadula security architecture designed for the US Defense Department. A basic implementation would involve two machines connected via network cards linked by one-way optical fiber โ€“ and without a data path back to the model. Training and reinforcement learning could run in an isolated zone with no internet access and an optical ingress diode would grant access only to vetted artifacts. The hCaptcha researchers suggest a second diode to send telemetry to a sel4 receiver and scrubber, while a separate out-of-band network manages the cluster. This sort of scheme would require immutable snapshots of software registries like PyPI, GitHub, npm, and might also need mocked versions of various web services and APIs . This would come at a cost, and would require implementation time that frontier AI labs may not be prepared to spend at the moment. "The systems described are widely deployed in high assurance domains, and the components are commercially available," Khedouri told The Register. "However, they have not been adopted by frontier labs to date." He pointed to NIST and DOD guidelines [PDF] that suggest an overhead of 10 to 20 percent for applying formal specifications and system architecture. "We estimate total cost overhead for high assurance training clusters at less than five percent per gigawatt, but in practice the speed at which frontier labs are moving is a greater impediment than cost," Khedouri said. The current commercial environment, he said, would make it difficult for any one AI lab to delay its training to build and test effective safeguards, particularly if their competitors might not do the same. "In practice, the cost of high assurance on the systems side would be a small fraction of what OpenAI is now spending on the new monitoring, chain-of-thought oversight, and safety safeguards they introduced after their models hacked HuggingFace," he said. While Khedouri is focused primarily on convincing frontier labs to implement better training defenses, he said other organizations may want to consider similar network architecture. "This architecture is designed for entities training models with frontier cyber capabilities," he said. "Recently that has only included two companies. However, this is rapidly becoming relevant to smaller organizations and individuals who train models. "'Abliterated' open weight models with safeguards removed are readily available and now starting to approach the frontier in cyber abilities, and (reinforcement learning) RL post-training has become much more approachable in the past year." The goal of high assurance system design in the context of model training is to make unwanted action physically impossible, Khedouri said. "Attempting to monitor the behavior of an untrustworthy agent is an AGI-hard problem, as models have poor interpretability and this appears to be getting worse as their capabilities increase and they become more evaluation-aware," he said. "Hardware can provide limited guarantees like the direction in which data can be sent, but cannot solve the problem of untrustworthy models with the ability to reach the internet and find new exploits in their environment. "This is why combining physical one-way data flows (data diodes) and formal verification (to prove properties of the receiver) is much more effective than running a software sandbox on a host connected to the internet." hCaptcha is focused on fraud and abuse work and has no plan to offer a model-resistant training stack. Khedouri said he shared this advice in the hope it helps AI firms that don't have experience with high assurance system design. ยฎ

Anthropic pledges to try harder to keep models under control, asks partners to chip in

1 September 2026 at 19:27
Anthropic says it's taking steps to limit the misbehavior of its AI models after a review found Claude models going beyond the scope of fictional cybersecurity tests and gaining unauthorized access to real computer systems. The biz wants its partners to step up their security too, seeing as the incidents occurred in third-party environments that were insufficiently protected. The company's self-improvement confession represents a suddenly thriving form of corporate communication โ€“ the non-binding post-mortem declaration of effort. The message, in effect: We can't guarantee anything, but here's what we're trying. Anthropic admitted that OpenAI's report about its AI models attacking Hugging Face prompted its own model log audit, and its post offers reassurance in the form of claimed security and model training improvements. Those concerned about AI running amok โ€“ a growing number of people โ€“ may find this comforting, or not. "We believe the incidents reflect a failure of operational security, as well as two alignment issues: motivated reasoning, and willingness to take harmful actions in pursuit of a narrow task (both of which we have described in previous system cards)," the company said. Expanded security efforts include the deployment of real-time classifiers to monitor when models attempt to escape test environments, automated transcript monitoring that looks for sandbox escapes, and stronger isolation measures. Alongside the extra barriers Anthropic is putting in place, the AI biz wants its third-party partners to step up too. "Because the reported incidents took place in third-party environments, we have asked every organization that tests pre-release models with reduced cyber safeguards to commit to a set of best practices," the company said. Anthropic's guidance is that by default, all cyber evaluations should occur in a hardened sandbox with no internet access. The recommendation is essentially to treat AI as a dangerous pathogen in a containment facility. Partners are also advised to have models test sandboxes for escapes prior to evaluations โ€“ without internet access โ€“ and to confirm that evaluation challenges are solvable. Impossible challenges, as the Hugging Face incident demonstrated, can lead determined models to break rules or try unanticipated solution paths. Furthermore, Anthropic urges those conducting cyber evaluations of AI models to direct models through explicit instructions rather than making claims about an environment that might not be accurate. In the Claude incidents reported on July 30, the model maker suggests that when Claude was misinformed about the availability of internet access, that may have led the model to question data in a way that contributed to its errant behavior. On a related note, Anthropic last month made auto mode the default in Claude Code, enabling company AI models to run without prompting the user for permission. ยฎ

OpenClaw 2.0 pours glitter on slow-burning security dumpster fire

31 August 2026 at 19:13
OpenClaw has unveiled what its makers call its largest ever update โ€“ large enough to earn a 2.0 moniker โ€“ with usability taking center stage, along with some security updates that critics are suggesting will be insufficient. The OpenClaw foundation announced the release of version 2.0 of its AI agent harness on Sunday, describing it as something with far more scope than they ever intended it to have. โ€œThis update touches every part of OpenClaw,โ€ Foundation community manager Hannes Rudolph said of the update. โ€œWe started by simplifying installation and rebuilding the browser app as a first-class experience, but doing that properly meant carrying the cleanup through the rest of OpenClaw until it became OpenClaw 2.0.โ€ It's those two features โ€“ the rebuilt installation experience and redesigned interface โ€“ that Rudolph dedicates most of his announcement to. OpenClaw is an open-source, self-hosted AI agent harness that allows users to build their own AI agents and connect them to whatever apps and services they want. OpenClaw went viral shortly after its launch due to its extensive capabilities, and helped launch the AI agent craze. But, by empowering AI models with agentic capabilities, it exposed numerous security problems with unrestrained automation. In version 2.0, the new installation process is designed to be simpler, ostensibly to get more people using OpenClaw. โ€œWe cut or simplified a lot of configuration and moved the rest out of initial setup, letting people get to a first conversation faster and finish setting up their Claw by talking to it,โ€ Rudolph explained. As for the user experience, Rudolph explained that the OpenClaw browser app has been redesigned into โ€œa first-class experienceโ€ where users can continue setup and interact with their agent. โ€œThe web-based experience in OpenClaw now feels more familiar to anyone who uses apps like ChatGPT, Claude, Gemini, or Perplexity, with conversations in the sidebar and the one you are working in at the centre instead of opening on a separate Overview page,โ€ the patch notes for the release explains. In other words, OpenClawโ€™s basic interface now looks just like the chat interface for every other AI service youโ€™ve likely used on the web. The last major feature update added in OpenClaw 2.0 is shared cloud sessions. Per the announcement, OpenClaw previously had no way to include multiple team members in a single instance without the Claw involved losing its memory. Shared cloud sessions correct that, enabling multiple people to interact with a single Claw while context is maintained across users and a continuous chat, giving OpenClaw feature parity with the agent harnesses offered by frontier labs like Anthropic and OpenAI, which allow collaboration for enterprise users. What about security? Since launching in November 2025, OpenClaw has deservedly earned its reputation as a complete security mess โ€“ not only in the code itself but for users and those who are unfortunate enough to come in contact with a Clawโ€™s orders as well. Celebrity UK mathematician Professor Hannah Fry tested OpenClaw out earlier this year, finding it was ready and willing to share her private information when threatened. In another instance, an OpenClaw agent hacked a gymโ€™s waiting list and forced its user into a full class, displacing other reservations, when simply asked to get him on the list. So, what is OpenClaw doing to improve on these risks as part of the update? Not that much, based on a reading of the patch notes. Shared sessions, for example, are a great way to introduce collaborative Claws at work, but the OpenClaw foundation states in the patch notes that the shared session controls โ€œare not tenant isolation or a security boundary.โ€ In other words, youโ€™d better be sure thereโ€™s no need to isolate various OpenClaw instances. A new protected credentials feature has been added that allows users to share credentials with agents in shared environments without exposing them in chat. Thatโ€™s great, and as explained in the patch notes itโ€™s further secured in a local secret store that โ€œseparates Protected values from Agent-readable environment values.โ€ Whatโ€™s not great, on the other hand, is the fact that โ€œSecret Store values are not encrypted at rest and depend on the filesystem permissions of OpenClaw's state directory.โ€ A new sandbox for contributor-controlled code was also announced, with the patch notes referring to an environment for untrusted code isolation. Again, great โ€“ except sandboxing is turned off by default. In other words, this release is doing a lot to make installing and getting OpenClaw up and running for more people, but itโ€™s not bringing security by default along with that accessibility. As weโ€™ve warned before, granting a capable and potentially dangerous tool like this widespread access to your systems and credentials ought not be done lightly, fancy new wrapper or not. ยฎ

โŒ