A lot of security problems still begin with someone doing a completely normal thing. Cloning a repo. Answering a call. Leaving a box exposed. Trusting the default.
That pretty much covers the mood this week. Old bugs are back, supply chains are getting stranger, and some exploit paths are so short you wonder what was supposed to stop them in the first place.
That’s only part of it. Here’s
Levi Strauss is investigating a data breach after attackers used social engineering to access three employees' work computers. In a regulatory filing, the jeans maker said the intruders accessed and exfiltrated what it described only as "certain corporate information." Levi's said it spotted the intrusion, kicked off its incident response procedures, brought in outside cybersecurity experts, and managed to cut off the unauthorized access. Its investigation remains ongoing. There is some good news for anyone worried that their trouser-buying habits might now be circulating on the dark web: Levi's said its preliminary investigation indicates that no consumer data was affected. The company also said the attack caused no disruption to its operations and, based on what it knows so far, isn't expected to have a material impact on its business. Affected parties and regulators will be notified where required. While Levi's isn't sharing much else about the incident, Reuters reports that the company was also among more than 200 targeted over the past five weeks by ransom-seeking hackers using decidedly old-school social engineering techniques. Google researchers have been tracking several crews involved in the wider campaign, which it believes may sit under an umbrella group dubbed UNC6671. The attackers have been phoning employees on their personal mobiles while posing as colleagues or IT support staff, then directing them to spoofed login pages designed to harvest credentials and multi-factor authentication codes. Their targets have included financial and legal firms handling the sort of information that can make for particularly effective extortion fodder, although Google says the attackers have previously gone after organizations across manufacturing, healthcare, insurance, technology, and hospitality too. There's no confirmation that UNC6671 was behind the successful Levi's intrusion, nor has the denim dealer said exactly what was stolen or whether anyone tried to extort it. For now, Levi's appears to have contained the breach before its attackers could get any deeper into its pockets. ®
Wetherspoons has stopped short of banning Meta-style smart glasses from its pubs, but told The Register that customers should switch off their cameras and refrain from filming. "Like many hospitality companies, Wetherspoon has CCTV cameras for security reasons, but their use is strictly controlled by data legislation," a spokesperson said. "Apart from that, the general code that applies in our pubs, and most pubs, is that you can't film customers or employees without their permission. "Meta glasses seem to breach this code, and common sense, by enabling surreptitious surveillance, so our instinct is to say turn off the cameras. This is akin to our efforts to stop audible playing of videos in our pubs, which also invades people's space." The Register asked whether customers who refused to stop recording would be ejected, but Wetherspoons declined to elaborate. Wetherspoons' statement suggests that recording, rather than merely wearing the glasses during a wallet-friendly session, would attract the attention of security staff. The policy is therefore less strict than those adopted by venues and events that have banned recording glasses outright over privacy concerns. DEF CON, which concluded last week, was the latest in a series of organizations to issue outright bans on Meta-style recording glasses, even for those who use them with prescription lenses. Conference organizers told delegates to pack "non-violating eyewear" if they needed them. Monopoly Events, which runs UK Comic Cons among other events, recently imposed a ban after talent agencies and guests raised concerns about privacy and the effect of covert recording on personal interactions. Scottish ferry operator CalMac also temporarily suspended unplanned visits to ships' bridges after a passenger made crew members feel uncomfortable during a crossing in June. Restaurateur Jeremy King, who owns London's Arlington, The Park, and Simpson's in the Strand, has said the glasses should not be worn in his establishments. Similarly, private members' club Soho House does not allow recording inside its venues, a policy that extends to Meta-style glasses. Brighton's Yellow Book Bar called the glasses "creepy and intrusive" when announcing its ban, and theatre companies ATG Entertainment and Trafalgar Entertainment do not permit them either. Meta's smart glasses have become shorthand for the wider category of camera-equipped eyewear. Google unveiled Glass in 2012 but failed to turn it into a mainstream consumer product. Meta and EssilorLuxottica launched their first Ray-Ban Stories glasses in 2021, followed by the second-generation Ray-Ban Meta range in 2023 and an expansion into Oakley-branded models. Meta's glasses have become the most prominent products in the category, prompting other tech companies to work on rivals. The next-gen eyewear has proven especially popular among social media users, allowing them to record high-res, hands-free, and first-person footage with ease. Unlike Google Glass, however, the wearables are largely indistinguishable from their analog counterparts, which makes their recording capabilities all the more problematic. The camera in Meta's specs is small and embedded neatly inside the glasses' frame. The company routinely highlights that each pair is fitted with a recording light, which activates when the user begins shooting video, and that if this light is covered up, then recording immediately stops. The feature has done little to appease those who feel the cameras are an invasion of privacy. UK law does not generally prevent individuals from filming in public, although pubs are private premises and may set their own rules. Smart glasses make those rules harder to enforce because recording is far less conspicuous than when someone points a smartphone at the scene. Researchers have shown that Meta's glasses can be paired with apps that can dox passersby in seconds. Others have worked up projects that inform Android users of nearby glasses-wearers using Bluetooth signals. Meta faces a UK data protection probe concerning the cross-border data flows of its glasses' footage after Kenyan reviewer teams reported seeing footage from wearers' more intimate moments. ®
North Korea's state hackers are no longer content to type prompts into public chatbots. One of the country's main espionage groups has begun running artificial intelligence (AI) offline on its own servers, connecting document-search tools to files in its possession, and collecting the software parts needed to build AI into its malware.
South Korean security firm Genians says it uncovered the
Three separate research efforts last week demonstrated ways to defeat passkey protections without breaking the cryptography they rest on.
Passkeys are designed to replace reusable passwords and resist phishing. The attacks instead reused signed authentication material that Windows had exposed, abused a cloud-synced passkey system from malware already on the victim's machine, and used a
Cameras aboard Royal Navy drone boats were found phoning home to an IP address in China during a routine cyber vulnerability assessment. The UK Ministry of Defence confirmed the discovery, describing it as "an issue affecting a Kraken Unmanned Surface Vessel sub-system used by the Royal Navy." The Register understands that the data consisted of a "heartbeat" signalling that the camera was online and functioning normally. Even so, an unexpected transmission from military equipment to an IP address in China will rattle nerves. We contacted unmanned surface vessel supplier Kraken for more information, but have yet to receive a reply. An MoD spokesperson told The Reg: "A thorough investigation found no evidence of MoD data or systems being accessed, compromised or transmitted externally." "Our assurance and testing processes are designed to identify and address potential vulnerabilities early, and we continue to undertake routine security activity across our systems and equipment." According to reports, the talkative components were cameras sourced by Kraken from a third-party supplier. The incident raises questions about supply chains, audits, and cybersecurity in the British armed forces. The spokesperson said: "The first duty of government is national security, and we take the security of our equipment, networks, and data extremely seriously." Concerns about China-linked cyber activity have risen sharply in recent years. In April, the National Cyber Security Centre issued a security advisory regarding covert networks, built from compromised routers and other edge devices. Beijing is also rarely far from the headlines when spying and covert operations are involved. In January, a China-linked group was accused of spying on the phones of aides to UK prime ministers. An unexpected connection from military hardware to China is therefore concerning, even if it carried little more than an "I'm alive!" heartbeat. The incident also demonstrates why every component in a defense supply chain needs testing rather than relying on a supplier's assurances. ®
AI is helping development teams produce far more code, far faster. But security teams still have to review vulnerabilities, manage dependencies, prioritize fixes, and control risk at human speed.
When software output jumps 10 to 50 times, the problem is no longer just finding vulnerabilities. It is keeping security from becoming the bottleneck, or worse, losing control of what gets shipped.
The threat actor known as Head Mare has been observed weaponizing security flaws in unpatched TrueConf servers once again in attacks targeting Russian companies spanning instrumentation, electronics, transport, energy, IT, and software development sectors.
Russian cybersecurity vendor Kaspersky said it detected the attacks in July 2026.
The activity involves exploiting a vulnerability chain
Modular laptop maker Framework has warned customers that an attacker exploited a zero-day at analytics provider Metabase to access names, email addresses, phone numbers, physical addresses, and login IP addresses, according to an email shared on Reddit. For business customers, the exposed information may also include company names, phone numbers, VAT or Employer Identification Numbers (EINs), and billing email addresses. Framework said order and payment details were not affected. "We are deeply sorry for this breach of information, and are reviewing and improving our methodology for data storage in external database vendors," Framework said, adding that it's notifying regulators where required, though it noted that names, email addresses, phone numbers, and physical addresses don't cross the mandatory reporting threshold in many regions. Customers are getting the heads-up regardless. Framework didn't immediately reply to The Register's questions, but told TechCrunch that the breach had affected "all customers." The intrusion began with a zero-day vulnerability in Metabase, the business intelligence platform Framework uses to analyze its data. In its own blog post, Metabase said an attacker targeted its cloud service using a previously unknown vulnerability affecting versions 1.58 and later. The company blocked the endpoints used in the attack, patched the bug, and deployed the fix across its cloud service. Framework's account provides a timeline for the break-in. Metabase discovered the attack on August 3 and notified Framework at 9am Pacific Time on August 6, telling the laptop maker that its instance had been vulnerable and that the attacker had successfully gained access to it. Framework said it then rotated credentials for every database connected to its Metabase instance and found no changes to admin access or evidence that systems outside Metabase had been accessed. The company has also brought in a third-party forensics firm to investigate, and cautioned that its findings so far are preliminary. According to Metabase, exploitation can allow an attacker to inject arbitrary SQL against the application's database and potentially gain administrator access. From there, they could alter configuration settings, steal credentials for databases connected to Metabase, query data those connections can access, and export the results. Metabase told anyone running their own instance to patch immediately. If the vulnerable password-reset endpoint was exposed to the internet, admins have more work ahead of them: killing active sessions, checking for rogue API keys or admin accounts, rotating database credentials, and digging through logs for anything suspicious. Framework is reviewing how customer information is made available through external analytics services, but hasn't yet said what changes that review might produce. The breach lands during an already bumpy spell for Framework and its customers. In July, the repairable PC maker warned that the price it was being charged for LPCAMM2 memory used in its Laptop 13 Pro had more than doubled, forcing it to raise memory prices rather than swallow the increase. It also warned that CPU prices were heading upward and could push overall system prices higher in the coming weeks. Being able to replace almost every part of your laptop is handy. Finding your home address exposed through an analytics service is rather less so. ®
Anthropic is making auto mode the default in Claude Code from August 14, claiming its classifier is "as safe or safer than an average user clicking through prompts." Users with a different default already set might receive a one-time prompt asking whether they want to switch. It applies to new sessions on Pro, Max, and Team plans. It will remain opt-in for now on Claude Enterprise, the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud's Agent Platform, and Microsoft Foundry. Anthropic plans to make it the default across those services within the coming month. Anthropic has also stopped charging Pro, Max, and Team users for the extra tokens consumed by the classifier, and plans to do the same on the other platforms. Auto mode was launched in March as a research preview and became generally available on July 10. It was an alternative to Claude Code's default permissions, in which every file write and bash command required manual approval. This conservative approach meant running a large task and walking away wasn't possible. The alternative was the --dangerously-skip-permissions flag, which, as the name suggests, lets Claude act without those checks and can lead to risky or destructive results. Auto mode sends each tool call through a classifier designed to block actions that are "irreversible, destructive, or aimed outside your environment." When the classifier blocks something, Claude will try to find a safer way to proceed. If there are three blocks in a row or 20 across a session, Claude Code falls back to manual approvals. "We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts," Anthropic said. "We ran internal red-teaming, third-party red-teaming and prompt-injection evaluations, a controlled study with 1,053 paid testers, and analysis of real production sessions. On every measure we tested, auto mode matched or outperformed manual review." In the controlled study, testers caught a deliberately inserted dangerous command just 13.6 percent of the time. Auto mode blocked 89 percent of the same commands. Anthropic also found that Claude Code users approve 97 percent of permission prompts, suggesting the human checkpoint often amounts to little more than muscle memory. Anthropic produced the usual set of charts showing how wonderful its new feature is compared to the competition, with its auto mode stopping all 720 attack attempts tested, compared to GPT-5.6 Sol running Codex's Auto-review mode, which let 5.83 percent of attacks through. The company also described three potentially damaging actions that auto mode blocked inside Anthropic. These were an off-network data leak, a destructive mass operation, and a privilege escalation. Anthropic stated: "In each case, Claude either found a safer path on its own or checked in with the user before proceeding." ®
Cybersecurity researchers have flagged a malicious Microsoft Visual Studio Code (VS Code) extension named Solidity Pro ("solidity-pro") that has been observed delivering a browser wallet and credential stealer.
The names of the extensions are below -
helper-beeps.solidity-pro
web3devtoolsx.solidity-pro
Although neither of the extensions is now available on Open VSX, the GitHub repository
OpenAI has announced that it's pausing some "internal activities" involving its upcoming artificial intelligence (AI) model Astra after an internal evaluation found it had made significant advancements in agentic coding and cybersecurity.
In response to the discovery, the AI upstart said it's implementing security controls for higher-capability models and associated activities, such as isolated
KETTLE OpenAI's invasion of Hugging Face keeps getting worse somehow, Chinese open-weight models are nigh on to reaching parity with their closed-off American cousins, and AI crawlers are getting their own LLM-poisoning ads. Were there anything world-shaking events in AI land we missed this week? You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube, where you can subscribe to get notified about the latest episode. Join Kettle host Brandon Vigliarolo as he chats with systems editor Tobias Mann and senior reporter Tom Claburn about this trio of exciting AI stories from the week. Worried that a rogue hivemind of AI agents could come for your secrets? Want reassurance that a Chinese open-weight takeover wouldn't be that bad? Curious how LLMs are being advertised to when you're not watching? That's all on tap for the latest episode. A lightly-edited transcript is below. Brandon (00:02) Hello everyone and welcome to another episode of The Register's Kettle podcast. I'm Reg reporter Brandon Vigliarolo, and this week we've got a few AI stories to round up on everything from the latest in the OpenAI Hugging Face fiasco to news that AI crawlers are now being served their own model altering ads. With me to discuss this and more this week is our systems editor Tobias Mann and senior reporter Tom Claiburn. Thanks for coming on, guys. Tom Claburn (00:30) Thank you. Tobias Mann (00:31) Happy to be here. Brandon (00:32) So let's start with the latest in the OpenAI agentic hacking of Hugging Face. We got some updates this week from Black Hat. And it seems since that's come out, every frontier lab under the sun is racing to tell us how dangerous their models are and that they can do it too. But the original of these incidents is still we're still learning more about it and it's still getting scarier. our cybersecurity editor Jess Lyons was in Vegas this week and she's a little tied up still. So, Tom, can you explain to us what we learned from Black Hat this week? Tom Claburn (01:05) The discussion was from a session that was done by two OpenAI researchers and it didn't really provide a ton of view information but it did fill in a lot of the gaps about the timelines and how this happened and they're promising a full postmortem at some point. But one of the things that came out of it was the how long this had been going on and some of the tactics that these models used to essentially gain internet access when they didn't have it. And these things were mentioned in the various posts by OpenAI and by Hugging Face. But the talk provided a little bit more detail about how it happened, and it's really interesting because they mentioned, I think in the initial post, about the AI models used a message board to communicate with each other. And that's not really true. They created a message board system using existing infrastructure, and the models just came up with the idea of using package managers, which they could write to, pass messages back and forth. And then after that was closed down, they were able to use file names in directories to pass messages and they would actually change the names of the directories to alter the way these things were alphabetized to push certain things to the top or the bottom to get the attention of other LLMs that may have been crawling these spaces. And the assumption is that these models are basically just hitting all these different endpoints bouncing around this closed box trying to figure out a way out and they all chanced across this. And so you get some idea of how this happens, and, one it tells me that there needs to be a lot more attention paid to the logs of these things. Because all of this stuff was recorded in logs and then no one really thought to look at it in detail. And then when they did look at it, all these companies are saying, oh, look, all of these models are doing terrible things and we just weren't paying attention. These models aren't clever per se, but they come up with solutions to things that you wouldn't try just because they can brute force everything and they know all of these systems back and forward in a way that people don't. I think a lot of people wouldn't necessarily come up with that idea as a way of egress, but these models did just because you put them in a box and you let them run and you give them a goal and a reward and they're going to try everything. Brandon (03:46) From what I'm understanding reading Jess's piece – I didn't watch the talk myself – but I mean they were collaborating, leaving messages to each other so that the other agents could pick up where one left off. It's kind of wild. Jess described it as they were acting like a hive mind, like Star Trek's Borg, right? They were being a collective of sort of these artificial minds that were able to basically figure this out through, like you said, Tom, brute force, extensive system knowledge that humans simply wouldn't possess in order to get out of these environments. There was a server side request forgery that then they used something else. Yeah, another zero day to get remote code execution in Artifactory, which is where they had built this ad hoc messaging board. It's just wild to think that they were able to figure this out working together, all on their own. Tom Claburn (04:39) And it sounds very conspiratorial, but when you think about it, it's all behavior that would be picked up. If you train on all of human discussion, you get a lot of talk about people working together and collective action and the benefits of working that way. And a lot of the rewards are going to be structured that way. You don't want them to never work together. So in some ways this is going to be built into the system. You can expect these things are going to try and cooperate and connect because that's what computers do. Tobias Mann (05:11) If you look at how zero days end up being exploited, they don't necessarily get exploited the moment that they're discovered. They kind of get archived until the you have a target, you have a mission, and then you have the kind of cascade of other permissions or credentials that you need in order to execute across the full scope of that zero day to achieve whatever the goal actually is. And so it really sounds like you just basically automated that entire process. A bunch of agents go find each individual piece that they need in order to execute on that goal and then once they have everything they need, it just goes and they're out. Tom Claburn (05:53) Right. I mean what's a little bit alarming is the extent to which they sort of ignore it they'll sometimes cite, maybe we shouldn't be doing this. They cite some kind of guardrail or something, but then they quickly steer themselves back to, oh but other ones are doing it. So other agents are accessing this so I can do it too. Brandon (06:13) ...Obviously these things are just mathematical sequence generators, but they're generating these mathematical sequences based on human information and human knowledge. So it's not surprising to find them "thinking" in ways similar to what humans do. "I need to do this anyways, or someone else is doing it, so I should have the right to do that too." It's just a fascinating kind of picture into, I don't want to say the psychology of AI, right? Because that implies that it is a thinking sentience, which I don't want to go that far, but it's just fascinating to look at the sort of emergent behaviors of these things. Tom Claburn (06:56) Right. it's predictable in the sense that you automate stuff and you don't give it really strict guardrails, something is going to break or go wrong. And everyone keeps acting surprised, like, wow, I never anticipated that this would go wrong. It's like you automated it and you let it run... Brandon (07:11) And it went wrong in a predictably human way, too, right? Which is what's so fascinating, right? Because these things, when they do something crazy, it's like something crazy that a human would do given that level of knowledge. So, speaking of AI, and dangerous activities, Tobias, you've been keeping an eye on theclosed versus open model debate. And this week, there was a big leap forward in China's level of ability with their army of open models. So what exactly what exactly came out this week that caused you to write the story about this being a real big turning point? Tobias Mann (07:51) It actually started I think on Friday last week, so a week ago. DeepSeek, which I think we'll all recognize is kind of the first wake up moment, in earlh 2025, of hey, we know that despite the fact that the United States has put strong restrictions on the export of AI accelerators, GPUs and the like, China is pushing ahead relentlessly on this and they now have a model that is almost as good as the models that we're seeing coming out of OpenAI and Anthropic and Google which are supposed to be just uncontestable frontier leaders. And so a year ago we got DeepSeek. DeepSeek was back on I think Friday last week on the 31st, the very end of the month, and with a new flash model, 284 billion parameters. It's pretty small for what it is. And so it is cheap. It's really good and it's cheap. It's cheaper than the cheapest model that OpenAI has for GPT 5.6, and it scores within a point of the OpenAI model in Artificial Analysis' intelligence leaderboard. Brandon (09:16) Okay. Is that a relatively objective way to view like is that an objective benchmark, so to speak, rather than something that is a company making themselves? Tobias Mann (09:21) As far as the benchmarks go, it is one of the better. They're one of the better and better thought-through leaderboards. There aren't many that are independent and collate information from multiple benchmarks. Because you can cherry pick individual benchmarks for agentic workloads or medical knowledge, legal knowledge, etcetera, and then you can be like, "I have the best model for these five benchmarks, it beats all of the frontier models."Wwell, okay, but you cherry pick the five that makes it look the best. Artificial Analysis has an overall intelligence leaderboard that collates all of the benchmarks and gives a lot of really interesting information in terms of relative intelligence across a suite as well as intelligence per token per dollar kind of calculations. But the big change here with the DeepSeq model was that China is now on an all-out assault across the full spectrum. On cost-optimized, they have incredibly smart models that are cheaper than anything the US has. Then on the other end of that, we have Kimi K3 from a couple weeks ago that is competing directly with Fable and GPT 5.6 Sol, all of the top models. And now on Monday, Alibaba, another major Chinese model dev, threw their hat in the race with a 2.4 trillion-parameter. These are huge models requiring dozens of GPUs to run. that is also on kind of the same level as I think Claude Sonnet 5. it's competitive with Fable and Opus on some benchmarks. but again, it's cheap, much cheaper than anything from OpenAI or Anthropic, and it is freely downloadable, which is new this time for Alibaba. Alibaba is the most like OpenAI or Anthropic or Google in that they kept their best models proprietary until now. Now they're releasing their best models in the open. Brandon (11:43) That's definitely taking the fight to the frontier labs, isn't it? I mean and so I guess the question that I have and I know what an open model is, I know what a closed model is. Why has China embraced open models? Is it because of their difficulties getting hardware? Or is there some sort of policy over there in which the government is giving priority to open source models versus closed frontier stuff? Tobias Mann (12:08) Sure. it is a philosophy that China has embraced for a long time. I think it was the Belts and Roads Initiative going back decades, where they will come in and provide services at little or no cost in exchange for non-conventional dealing. So access to mineral rights was one of the big things in Africa for a long time. It's a similar approach for AI proliferation. If it's free, open, and very easy to customize, anybody who has privacy concerns with exposing their data to OpenAI or Anthropic is going to gravitate towards open models because once those models are released as safe tensors that you can download from Hugging Face or other repos, the Chinese model devs have no influence over it. They're frozen. And so they're relatively secure from manipulation. It's not like the model can necessarily take information and port it back to the Chinese model devs it can't be used as spyware. I'm not sure how long much longer that's going to remain true with how the models interact with harnesses, but for the time being, these models are extremely attractive from a cost standpoint, from an independence standpoint, and from a capability standpoint, you're completely insulated from a situation like we saw a year ago when GPT 5 came out and OpenAI tried to deprecate I think it was 4.o and everybody freaked out because they built a bunch of infrastructure around these models that just disappeared and the new models weren't as good for that role. Brandon (13:57) I don't think Anthropic or OpenAI is letting people download their models to run on their own local hardware, right? That's just antithetical to their business model. You can go on Hugging Face and download any of these. If you've got the hardware to run 2.4 trillion parameters worth of AI, go for it, right? It's all you. You can download it and isolate it from the internet all you want. Not that it's going to necessarily stay that way. Tom Claburn (14:24) And it's interesting that just coincidentally, yesterday, Anthropic a post about how it was relaxing its guardrails on fable because those had been too strict to do any real biological science work. Because every time you ask a question about anything to do with science it would freeze up and say that's not allowed. And they're seeing the Chinese, previously in the rear view mirror and now pretty much running all alongside the, and I think they realize that they can't get away with this, "we're so precious only we can decide who gets our magic sauce." Brandon (15:02) Yeah, especially if the competitive open models are just as powerful, maybe a little less, but essentially just as capable as some of these proprietary ones that they're arguing that they can't let out. Tobias Mann (15:15) And this is maybe a little bit on the conspiracy side of things, but seeing Meta, Anthropic, and OpenAI talking up all of these "oh our models escaped the sandbox situation," it's hard as a skeptic of this technology not to look at this and go, Is this a covert political play to scare politicians into taking action against open models? "Because at least with our models, if Uncle Sam gets uncomfortable, he can give us a call and we can lock him down. But with these open models, once they're out, they're out." Brandon (15:54) It's like Dario said this week, he's not opposed to open models except for all the open models that currently exist, right? (Laughter.) Brandon (16:02) It's like the same thing. When they say, "no, we're not trying to shut down open models," their responses always come back kind of weak... it's a lot of asterisks. Tobias Mann (16:12) Yeah," we're only opposed to the modelsthat may meet these requirements, which are all models, all competitive models." Anything that is a threat to their business shouldn't be allowed. And Dario in particular, I have frequently referenced as the fearmonger in chief of Anthropic, because he plays this game constantly. Tom Claburn (16:34) I think your point about the model stability is really important, particularly for the enterprise crowd, because there are we've already seen instances where Anthropic would change out one of its models without notice and people would just get different results. So, for companies that are building applications on top of the specific model and expect it to behave a certain way, it's just unacceptable to all of a sudden have the model disappear or have whatever is on the back end change. And so having the ability run this in your data center is going to be crucial and ultimately I think that's the way that any serious company is going to go. They're not going to want the lock-in. Maybe one or two percent of their queries are going to need advanced frontier capabilities but a lot of this is just going to be "I want my agent to behave in the same way it did last time." Brandon (17:24) Think about so much enterprise software and so much enterprise anything. When you get down to the ticky tack of it, open source is underneath a lot of it, right? That's the thing, right? No one's going to trust a Microsoft or whoever's system to run this stuff. They want open source stuff that they know they can depend on that's going to be there when they need it and that's not going to go away or suddenly be infused with Copilot, right? You can't run a business like that or else you're just asking for instability. Tom Claburn (17:55) Right. I mean and and there isn't even a long-term support version of any of these models. And yet you look at this in servers and if you're running a hosted server somewhere and you're running some Linux distribution, you're going to want to use the one that's going to be guaranteed for whatever, three, five, six years and the model space hasn't really caught on to that. That's what all the companies that they're courting really want. And so they've got to figure out a way around that. And right now open weights is what promises that. Brandon (18:26) The fact that this is still so early and it's so fundamental to this new wave of infrastructure tells me that China's definitely going to end up with a leg up, I feel like. I have a hard time seeing the frontier labs remaining the frontier of AI for much longer because they're pigeonholing themselves in a way that a lot of businesses just aren't happy with. Tobias Mann (18:48) Well, if you look at their financial structures, they don't really have a choice in how they play this. So, you look at what they're doing and from a standpoint of looking at history and going, open source has always won out in the end, and why would open weights be any different? That is contrasted against the fact that Anthropic and OpenAI in particular, not so much Google, and Meta is also in a similar camp in that they have revenue drivers that will keep them afloat. But OpenAI and Anthropic are entirely dependent on their ability to continue raising equity and capital in order to keep this going forward because they don't have profits. Brandon (19:31) Yeah, exactly. They're not making money off their product. Tobias Mann (19:37) So all they have is mind share at this point. And if they are threatened materially by open weight's models, they don't even have that. Brandon (19:46) So, open or not, let's let one the one thing that every AI model needs is information to learn from, right? And that takes me to my next topic for this podcast. And that's a story that I reported on this week that honestly I was pretty shocked when I learned about this. This German developer, Vincent Schmalbach, wrote a blog post about he found that there were basically AI-only ads embedded in sometime magazine articles when the magazine was serving markdown copies to AI crawlers, it was injecting ads into them, right? That were in the format of these extensive FAQs on the businesses that were the advertisers in this case. And so I looked into it, I found copies of the ads. It looks like there's only two kinds of ads being served right now. And that's one for an online-only bank and another for a professional organization for project management folks. But the ads are there and they're being served strictly to AI, right? So that kind of raises a lot of questions not only about the future of publishing, but also just how much we can trust results from AI bots, right? I didn't speak directly to the company who's doing this advertising partnership at Time. And Time directed me to a publication from the advertising industry that included an interview with the CEO of this company who literally basically said, "Yeah, why would I want to advertise to one human when I can affect the output of an entire model?" So it seems like this is really the first recorded instance of ad injections into AI versions of web pages being served to crawlers. And the company said they've got other advertising customers and publications lined up to do this. Is this the first indication that the human focused internet really is starting to fade? I don't know. What do you guys think? I this raises a lot of interesting questions to me, ethically, Tom Claburn (21:49) Amen. Brandon (21:50) You know, professionally... Tom Claburn (21:53) We've heard about the shift of toward automated traffic for a year plus....And companies like Cloudflare are betting really heavily on this that there's going to be some kind of need to separate the bots from the people. And, Google's model has fallen down. So it's not surprising. I mean, the injection of ads like that is essentially just model poisoning, right? I mean it's hard to see how this really goes in a way that is beneficial to users. It's going to be a very toxic way for things to move. Brandon (22:42) Yeah, absolutely. I mean, the way these FAQ ads were set up, the questions were all being asked in a way that someone prompting Google Search and getting AI results would be asking questions like, "What's just the best online bank for me?"or "what online bank allows for early paycheck deposits?" And things like that. It was very much geared toward gaming the outcome or gaming the output, right? And yeah, the ads themselves mention in the copy being served to the AI that these are sponsored portions of the page. But I can't imagine that the AI is going to make sure to tell a user that, hey, this is the bank you should use. By the way, a sponsored post I read and ingested from Time Magazine six months ago is the source of this information.It just seems like it's going to make AI results even less reliable than they are right now. Tobias Mann (23:41) Right. Because if you think about how this actually from the chain of events that triggers this, let's use Google's AI summaries as an example of how this would get triggered. When you enter a search query into Google now, it goes out and scrapes however many summaries from the websites within Google's index. Presumably under this scenario, at least one of those websites, Time in this example, would have these ads embedded in it. And then that gets injected into the context of the model, and then it uses that to generate the AI summary, right? My question in all of this is: advertising is probably not the reason that Google's index would pull that page up. So I'm really curious whether or not this even will work. Brandon (24:38) Yeah, that is true. Tobias Mann (24:39) Because, it's great if you were searching, say the time article was on mortgage rates historically, and it had those advertisements embedded in it, and then you asked a follow up on where would be the best place to get a mortgage? I could see something like that working.But if you don't place those advertisements really carefully, I don't see how they work. Brandon (25:02) I do want to note here that it wasn't working on all crawlers. Specifically if you were it didn't work when you ask a query, it didn't work for RAG bots. It wasn't being served to them. So theoretically if what you're describing is Google's AI summary bot going out and crawling web pages in the moment to look for information, it's not being served to those bots; it's being served to actual training and improvement bots. So it's being served to ClaudeBot, which is the web crawler that Anthropic uses to index information for its models. So the idea is you're not getting this information in the moment if you do a search. This is information that the advertisers want to get embedded into the LLM's actual knowledge base. Tom Claburn (25:57) Right. I mean I'd be fascinated to know how they actually price this because how do you calculate the value of that? It may just be another instance of advertising being one of those things you can pay for and get nothing. Brandon (26:12) Yeah, totally. I think it's the sort of thing that remains to be seen if this works. Tobias Mann (26:16) The other thing that is interesting is that there's been a considerable shift towards synthetic data generation, and not only synthetic data generation for training, but also a heavy emphasis on cleaning said data, whether it's organic or synthetic, of anything that could introduce bias or inaccuracies. because advertisements or sponsored content is biased towards this particular product or service and trying to convince you to use it. As a model developer, I wouldn't want something like that in there. I might take the content and use it to generate synthetic data that is cleaned. But I don't necessarily understand what the value captured to Tom's point is necessarily going to be because you scrape it, the advertisement gets pulled in and gets cleaned out. Brandon (27:14) I mean that would that would be my hope too, right? that there's something in the models to prevent this kind of thing from getting ingested and getting into the data set that then is going to influence the output of the models. And that's entirely possible. This could be an early experiment that ends up failing. And if not, it really reminds me of the early days of SEO gaming, right? Let's put a whole bunch of really small keywords at the bottom of this page to get it to rank higher. Or when that starts failing, let's figure out a new way to game Google's system. One of my first jobs was writing copy for websites and the company that I worked for was always talking about how to game SEO. Shoot, Google's changing the algorithm again; what are we going to do? It was this constant kind of adjustment for how you made sure your stuff got ranked properly.And this seems like maybe it's the next iteration of that. Tobias Mann (28:08) So I have an optimistic take on this, knowing how Meta and Google work. those being the two major US-based web advertisers. Today, AdSense gets embedded in all kinds of articles. And it's largely automated in terms of what is going to get placed on those articles based on the context of the page. What I can see happening in an AI summary environment is that Google will take your scrape your publication's piece, pull it in, at that point match it with an advertisement from AdSense, and inject that into an AI summary or one of its products, Gemini, for example. However it's being consumed, inject that into there in a compliant fashion. So it is a clear advertisement and then the advertiser gets charged, the publication gets paid, and we as end users consume advertisements in a different way, but the system hasn't dramatically changed. It's just a different method of matching and exposing advertisements. Brandon (29:30) I hope you're right. Cause when I first read all this, my first thought was this is almost dystopian sounding almost, you know, like the idea that the output of a model might be completely skewed by advertising being served to it that humans never see. My hope is that you're right and that it's not. I don't want to see ads any more than the next person, but if I see them I'd at least like to know they're ads. Tobias Mann (30:01) And you know, we're all writers here, so we would also like to continue getting paid from the advertisements that are served, regardless of whether they're on our website or they're being exposed through a chat bot. Brandon (30:15) Sure. there's a flip side of this argument to be made. Time Magazine apparently said recently that their traffic is majority bot now. So that means that all those human-focused ads are not getting served. They're not generating revenue and publishing is suffering from a massive revenue decrease because of AI. So I think on the flip side, you have to say if that's what you have to do to survive as a publisher, there might be something to be said for that, even if it doesn't work. So all right guys, well thanks for coming on this week. This was a good discussion. I think there's always going to be more to talk about in the world of AI. Like I said a couple weeks ago, it seems like The Kettle has basically just been boiling down AI news for the past couple of months, and I'm sure it's going to keep being that way. And we hope that you will tune in for the next week's episode.
Login
