Why CISA is Warning CISOs About a Breach at Sisense

The U.S. Cybersecurity and Infrastructure Security Agency (CISA) said today it is investigating a breach at business intelligence company Sisense, whose products are designed to allow companies to view the status of multiple third-party online services in a single dashboard. CISA urged all Sisense customers to reset any credentials and secrets that may have been shared with the company, which is the same advice Sisense gave to its customers Wednesday evening.

New York City based Sisense has more than a thousand customers across a range of industry verticals, including financial services, telecommunications, healthcare and higher education. On April 10, Sisense Chief Information Security Officer Sangram Dash told customers the company had been made aware of reports that “certain Sisense company information may have been made available on what we have been advised is a restricted access server (not generally available on the internet.)”

“We are taking this matter seriously and promptly commenced an investigation,” Dash continued. “We engaged industry-leading experts to assist us with the investigation. This matter has not resulted in an interruption to our business operations. Out of an abundance of caution, and while we continue to investigate, we urge you to promptly rotate any credentials that you use within your Sisense application.”

In its alert, CISA said it was working with private industry partners to respond to a recent compromise discovered by independent security researchers involving Sisense.

“CISA is taking an active role in collaborating with private industry partners to respond to this incident, especially as it relates to impacted critical infrastructure sector organizations,” the sparse alert reads. “We will provide updates as more information becomes available.”

Sisense declined to comment when asked about the veracity of information shared by two trusted sources with close knowledge of the breach investigation. Those sources said the breach appears to have started when the attackers somehow gained access to the company’s Gitlab code repository, and in that repository was a token or credential that gave the bad guys access to Sisense’s Amazon S3 buckets in the cloud.

Customers can use Gitlab either as a solution that is hosted in the cloud at Gitlab.com, or as a self-managed deployment. KrebsOnSecurity understands that Sisense was using the self-managed version of Gitlab.

Both sources said the attackers used the S3 access to copy and exfiltrate several terabytes worth of Sisense customer data, which apparently included millions of access tokens, email account passwords, and even SSL certificates.

The incident raises questions about whether Sisense was doing enough to protect sensitive data entrusted to it by customers, such as whether the massive volume of stolen customer data was ever encrypted while at rest in these Amazon cloud servers.

It is clear, however, that unknown attackers now have all of the credentials that Sisense customers used in their dashboards.

The breach also makes clear that Sisense is somewhat limited in the clean-up actions that it can take on behalf of customers, because access tokens are essentially text files on your computer that allow you to stay logged in for extended periods of time — sometimes indefinitely. And depending on which service we’re talking about, it may be possible for attackers to re-use those access tokens to authenticate as the victim without ever having to present valid credentials.

Beyond that, it is largely up to Sisense customers to decide if and when they change passwords to the various third-party services that they’ve previously entrusted to Sisense.

Earlier today, a public relations firm working with Sisense reached out to learn if KrebsOnSecurity planned to publish any further updates on their breach (KrebsOnSecurity posted a screenshot of the CISO’s customer email to both LinkedIn and Mastodon on Wednesday evening). The PR rep said Sisense wanted to make sure they had an opportunity to comment before the story ran.

But when confronted with the details shared by my sources, Sisense apparently changed its mind.

“After consulting with Sisense, they have told me that they don’t wish to respond,” the PR rep said in an emailed reply.

Update, 6:49 p.m., ET: Added clarification that Sisense is using a self-hosted version of Gitlab, not the cloud version managed by Gitlab.com.

Also, Sisense’s CISO Dash just sent an update to customers directly. The latest advice from the company is far more detailed, and involves resetting a potentially large number of access tokens across multiple technologies, including Microsoft Active Directory credentials, GIT credentials, web access tokens, and any single sign-on (SSO) secrets or tokens.

The full message from Dash to customers is below:

“Good Afternoon,

We are following up on our prior communication of April 10, 2024, regarding reports that certain Sisense company information may have been made available on a restricted access server. As noted, we are taking this matter seriously and our investigation remains ongoing.

Our customers must reset any keys, tokens, or other credentials in their environment used within the Sisense application.

Specifically, you should:
– Change Your Password: Change all Sisense-related passwords on http://my.sisense.com
– Non-SSO:
– Replace the Secret in the Base Configuration Security section with your GUID/UUID.
– Reset passwords for all users in the Sisense application.
– Logout all users by running GET /api/v1/authentication/logout_all under Admin user.
– Single Sign-On (SSO):
– If you use SSO JWT for the user’s authentication in Sisense, you will need to update sso.shared_secret in Sisense and then use the newly generated value on the side of the SSO handler.
– We strongly recommend rotating the x.509 certificate for your SSO SAML identity provider.
– If you utilize OpenID, it’s imperative to rotate the client secret as well.
– Following these adjustments, update the SSO settings in Sisense with the revised values.
– Logout all users by running GET /api/v1/authentication/logout_all under Admin user.
– Customer Database Credentials: Reset credentials in your database that were used in the Sisense application to ensure continuity of connection between the systems.
– Data Models: Change all usernames and passwords in the database connection string in the data models.
– User Params: If you are using the User Params feature, reset them.
– Active Directory/LDAP: Change the username and user password of users whose authorization is used for AD synchronization.
– HTTP Authentication for GIT: Rotate the credentials in every GIT project.
– B2D Customers: Use the following API PATCH api/v2/b2d-connection in the admin section to update the B2D connection.
– Infusion Apps: Rotate the associated keys.
– Web Access Token: Rotate all tokens.
– Custom Email Server: Rotate associated credentials.
– Custom Code: Reset any secrets that appear in custom code Notebooks.

If you need any assistance, please submit a customer support ticket at https://community.sisense.com/t5/support-portal/bd-p/SupportPortal and mark it as critical. We have a dedicated response team on standby to assist with your requests.

At Sisense, we give paramount importance to security and are committed to our customers’ success. Thank you for your partnership and commitment to our mutual security.

Regards,

Sangram Dash
Chief Information Security Officer”

Juniper Support Portal Exposed Customer Device Info

Until earlier this week, the support website for networking equipment vendor Juniper Networks was exposing potentially sensitive information tied to customer products, including which devices customers bought, as well as each product’s warranty status, service contracts and serial numbers. Juniper said it has since fixed the problem, and that the inadvertent data exposure stemmed from a recent upgrade to its support portal.

Sunnyvale, Calif. based Juniper Networks makes high-powered Internet routers and switches, and its products are used in some of the world’s largest organizations. Earlier this week KrebsOnSecurity heard from a reader responsible for managing several Juniper devices, who found he could use Juniper’s customer support portal to find device and support contract information for other Juniper customers.

Logan George is a 17-year-old intern working for an organization that uses Juniper products. George said he found the data exposure earlier this week by accident while searching for support information on a particular Juniper product.

George discovered that after logging in with a regular customer account, Juniper’s support website allowed him to list detailed information about virtually any Juniper device purchased by other customers. Searching on Amazon.com in the Juniper portal, for example, returned tens of thousands of records. Each record included the device’s model and serial number, the approximate location where it is installed, as well as the device’s status and associated support contract information.

George said the exposed support contract information is potentially sensitive because it shows which Juniper products are most likely to be lacking critical security updates.

“If you don’t have a support contract you don’t get updates, it’s as simple as that,” George said. “Using serial numbers, I could see which products aren’t under support contracts. And then I could narrow down where each device was sent through their serial number tracking system, and potentially see all of what was sent to the same location. A lot of companies don’t update their switches very often, and knowing what they use allows someone to know what attack vectors are possible.”

In a written statement, Juniper said the data exposure was the result of a recent upgrade to its support portal.

“We were made aware of an inadvertent issue that allowed registered users to our system to access serial numbers that were not associated with their account,” the statement reads. “We acted promptly to resolve this issue and have no reason to believe at this time that any identifiable or personal customer data was exposed in any way. We take these matters seriously and always use these experiences to prevent further similar incidents. We are actively working to determine the root cause of this defect and thank the researcher for bringing this to our attention.”

The company has not yet responded to requests for information about exactly when those overly permissive user rights were introduced. However, the changes may date back to September 2023, when Juniper announced it had rebuilt its customer support portal.

George told KrebsOnSecurity the back-end for Juniper’s support website appears to be supported by Salesforce, and that Juniper likely did not have the proper user permissions established on its Salesforce assets. In April 2023, KrebsOnSecurity published research showing that a shocking number of organizations — including banks, healthcare providers and state and local governments — were leaking private and sensitive data thanks to misconfigured Salesforce installations.

Nicholas Weaver, a researcher at University of California, Berkeley’s International Computer Science Institute (ICSI) and lecturer at UC Davis, said the complexity layered into modern tech support portals leaves much room for error.

“This is a reminder of how hard it is to build these large systems like support portals, where you need to be able to manage gazillions of users with distinct access roles,” Weaver said. “One minor screw up there can produce hilarious results.”

Last month, computer maker Hewlett Packard Enterprise announced it would buy Juniper Networks for $14 billion, reportedly to help beef up the 100-year-old technology company’s artificial intelligence offerings.

Update, 11:01 a.m. ET: An earlier version of this story quoted George as saying he was able to see support information for the U.S. Department of Defense. George has since clarified that while one block of device records he found was labeled “Department of Defense,” that record appears to belong to a different country.

LastPass: ‘Horse Gone Barn Bolted’ is Strong Password

The password manager service LastPass is now forcing some of its users to pick longer master passwords. LastPass says the changes are needed to ensure all customers are protected by their latest security improvements. But critics say the move is little more than a public relations stunt that will do nothing to help countless early adopters whose password vaults were exposed in a 2022 breach at LastPass.

LastPass told customers this week they would be forced to update their master password if it was less than 12 characters. LastPass officially instituted this change back in 2018, but some undisclosed number of the company’s earlier customers were never required to increase the length of their master passwords.

This is significant because in November 2022, LastPass disclosed a breach in which hackers stole password vaults containing both encrypted and plaintext data for more than 25 million users.

Since then, a steady trickle of six-figure cryptocurrency heists targeting security-conscious people throughout the tech industry has led some security experts to conclude that crooks likely have succeeded at cracking open some of the stolen LastPass vaults.

KrebsOnSecurity last month interviewed a victim who recently saw more than three million dollars worth of cryptocurrency siphoned from his account. That user signed up with LastPass nearly a decade ago, stored their cryptocurrency seed phrase there, and yet never changed his master password — which was just eight characters. Nor was he ever forced to improve his master password.

That story cited research from Adblock Plus creator Wladimir Palant, who said LastPass failed to upgrade many older, original customers to more secure encryption protections that were offered to newer customers over the years.

For example, another important default setting in LastPass is the number of “iterations,” or how many times your master password is run through the company’s encryption routines. The more iterations, the longer it takes an offline attacker to crack your master password.

Palant said that for many older LastPass users, the initial default setting for iterations was anywhere from “1” to “500.” By 2013, new LastPass customers were given 5,000 iterations by default. In February 2018, LastPass changed the default to 100,100 iterations. And very recently, it upped that again to 600,000. Still, Palant and others impacted by the 2022 breach at LastPass say their account security settings were never forcibly upgraded.

Palant called this latest action by LastPass a PR stunt.

“They sent this message to everyone, whether they have a weak master password or not – this way they can again blame the users for not respecting their policies,” Palant said. “But I just logged in with my weak password, and I am not forced to change it. Sending emails is cheap, but they once again didn’t implement any technical measures to enforce this policy change.”

Either way, Palant said, the changes won’t help people affected by the 2022 breach.

“These people need to change all their passwords, something that LastPass still won’t recommend,” Palant said. “But it will somewhat help with the breaches to come.”

LastPass CEO Karim Toubba said changing master password length (or even the master password itself) is not designed to address already stolen vaults that are offline.

“This is meant to better protect customers’ online vaults and encourage them to bring their accounts up to the 2018 LastPass standard default setting of a 12-character minimum (but could opt out from),” Toubba said in an emailed statement. “We know that some customers may have chosen convenience over security and utilized less complex master passwords despite encouragement to use our (or others) password generator to do otherwise.”

A basic functionality of LastPass is that it will pick and remember lengthy, complex passwords for each of your websites or online services. To automatically populate the appropriate credentials at any website going forward, you simply authenticate to LastPass using your master password.

LastPass has always emphasized that if you lose this master password, that’s too bad because they don’t store it and their encryption is so strong that even they can’t help you recover it.

But experts say all bets are off when cybercrooks can get their hands on the encrypted vault data itself — as opposed to having to interact with LastPass via its website. These so-called “offline” attacks allow the bad guys to conduct unlimited and unfettered “brute force” password cracking attempts against the encrypted data using powerful computers that can each try millions of password guesses per second.

A chart on Palant’s blog post offers an idea of how increasing password iterations dramatically increases the costs and time needed by the attackers to crack someone’s master password. Palant said it would take a single high-powered graphics card about a year to crack a password of average complexity with 500 iterations, and about 10 years to crack the same password run through 5,000 iterations.

However, these numbers radically come down when a determined adversary also has other large-scale computational assets at their disposal, such as a bitcoin mining operation that can coordinate the password-cracking activity across multiple powerful systems simultaneously.

Meaning, LastPass users whose vaults were never upgraded to higher iterations and whose master passwords were weak (less than 12 characters) likely have been a primary target of distributed password-cracking attacks ever since the LastPass user vaults were stolen late last year.

Asked why some LastPass users were left behind on older security minimums, Toubba said a “small percentage” of customers had corrupted items in their password vaults that prevented those accounts from properly upgrading to the new requirements and settings.

“We have been able to determine that a small percentage of customers have items in their vaults that are corrupt and when we previously utilized automated scripts designed to re-encrypt vaults when the master password or iteration count is changed, they did not complete,” Toubba said. “These errors were not originally apparent as part of these efforts and, as we have discovered them, we have been working to be able to remedy this and finish the re-encryption.”

Nicholas Weaver, a researcher at University of California, Berkeley’s International Computer Science Institute (ICSI) and lecturer at UC Davis, said LastPass made a huge mistake years ago by not force-upgrading the iteration count for existing users.

“And now this is blaming the users — ‘you should have used a longer passphrase’ — not them for having weak defaults that were never upgraded for existing users,” Weaver said. “LastPass in my book is one step above snake-oil. I used to be, ‘Pick whichever password manager you want,’ but now I am very much, ‘Pick any password manager but LastPass.'”

Asked why LastPass isn’t recommending that users change all of the passwords secured by the encrypted master password that was stolen when the company got hacked last year, Toubba said it’s because “the data demonstrates that the majority of our customers follow our recommendations (or greater), and the probability of successfully brute forcing vault encryption is greatly reduced accordingly.”

“We’ve been telling customers since December of 2022 that they should be following recommended guidelines,” Toubba continued. “And if they haven’t followed the guidelines we recommended that they change their downstream passwords.”

Experts Fear Crooks are Cracking Keys Stolen in LastPass Breach

In November 2022, the password manager service LastPass disclosed a breach in which hackers stole password vaults containing both encrypted and plaintext data for more than 25 million users. Since then, a steady trickle of six-figure cryptocurrency heists targeting security-conscious people throughout the tech industry has led some security experts to conclude that crooks likely have succeeded at cracking open some of the stolen LastPass vaults.

Taylor Monahan is lead product manager of MetaMask, a popular software cryptocurrency wallet used to interact with the Ethereum blockchain. Since late December 2022, Monahan and other researchers have identified a highly reliable set of clues that they say connect recent thefts targeting more than 150 people. Collectively, these individuals have been robbed of more than $35 million worth of crypto.

Monahan said virtually all of the victims she has assisted were longtime cryptocurrency investors, and security-minded individuals. Importantly, none appeared to have suffered the sorts of attacks that typically preface a high-dollar crypto heist, such as the compromise of one’s email and/or mobile phone accounts.

“The victim profile remains the most striking thing,” Monahan wrote. “They truly all are reasonably secure. They are also deeply integrated into this ecosystem, [including] employees of reputable crypto orgs, VCs [venture capitalists], people who built DeFi protocols, deploy contracts, run full nodes.”

Monahan has been documenting the crypto thefts via Twitter/X since March 2023, frequently expressing frustration in the search for a common cause among the victims. Then on Aug. 28, Monahan said she’d concluded that the common thread among nearly every victim was that they’d previously used LastPass to store their “seed phrase,” the private key needed to unlock access to their cryptocurrency investments.

Armed with your secret seed phrase, anyone can instantly access all of the cryptocurrency holdings tied to that cryptographic key, and move the funds to anywhere they like.

Which is why the best practice for many cybersecurity enthusiasts has long been to store their seed phrases either in some type of encrypted container — such as a password manager — or else inside an offline, special-purpose hardware encryption device, such as a Trezor or Ledger wallet.

“The seed phrase is literally the money,” said Nick Bax, director of analytics at Unciphered, a cryptocurrency wallet recovery company. “If you have my seed phrase, you can copy and paste that into your wallet, and then you can see all my accounts. And you can transfer my funds.”

Bax said he closely reviewed the massive trove of cryptocurrency theft data that Taylor Monahan and others have collected and linked together.

“It’s one of the broadest and most complex cryptocurrency investigations I’ve ever seen,” Bax said. “I ran my own analysis on top of their data and reached the same conclusion that Taylor reported. The threat actor moved stolen funds from multiple victims to the same blockchain addresses, making it possible to strongly link those victims.”

Bax, Monahan and others interviewed for this story say they’ve identified a unique signature that links the theft of more than $35 million in crypto from more than 150 confirmed victims, with roughly two to five high-dollar heists happening each month since December 2022.

KrebsOnSecurity has reviewed this signature but is not publishing it at the request of Monahan and other researchers, who say doing so could cause the attackers to alter their operations in ways that make their criminal activity more difficult to track.

But the researchers have published findings about the dramatic similarities in the ways that victim funds were stolen and laundered through specific cryptocurrency exchanges. They also learned the attackers frequently grouped together victims by sending their cryptocurrencies to the same destination crypto wallet.

By identifying points of overlap in these destination addresses, the researchers were then able to track down and interview new victims. For example, the researchers said their methodology identified a recent multi-million dollar crypto heist victim as an employee at Chainalysis, a blockchain analysis firm that works closely with law enforcement agencies to help track down cybercriminals and money launderers.

Chainalysis confirmed that the employee had suffered a high-dollar cryptocurrency heist late last month, but otherwise declined to comment for this story.

Bax said the only obvious commonality between the victims who agreed to be interviewed was that they had stored the seed phrases for their cryptocurrency wallets in LastPass.

“On top of the overlapping indicators of compromise, there are more circumstantial behavioral patterns and tradecraft which are also consistent between different thefts and support the conclusion,” Bax told KrebsOnSecuirty. “I’m confident enough that this is a real problem that I’ve been urging my friends and family who use LastPass to change all of their passwords and migrate any crypto that may have been exposed, despite knowing full well how tedious that is.”

LastPass declined to answer questions about the research highlighted in this story, citing an ongoing law enforcement investigation and pending litigation against the company in response to its 2022 data breach.

“Last year’s incident remains the subject of an ongoing investigation by law enforcement and is also the subject of pending litigation,” LastPass said in a written statement provided to KrebsOnSecurity. “Since last year’s attack on LastPass, we have remained in contact with law enforcement and continue to do so.”

Their statement continues:

“We have shared various technical information, Indicators of Compromise (IOCs), and threat actor tactics, techniques, and procedures (TTPs) with our law enforcement contacts as well as our internal and external threat intelligence and forensic partners in an effort to try and help identify the parties responsible. In the meantime, we encourage any security researchers to share any useful information they believe they may have with our Threat Intelligence team by contacting securitydisclosure@lastpass.com.”

THE LASTPASS BREACH(ES)

On August 25, 2022, LastPass CEO Karim Toubba wrote to users that the company had detected unusual activity in its software development environment, and that the intruders stole some source code and proprietary LastPass technical information. On Sept. 15, 2022, LastPass said an investigation into the August breach determined the attacker did not access any customer data or password vaults.

But on Nov. 30, 2022, LastPass notified customers about another, far more serious security incident that the company said leveraged data stolen in the August breach. LastPass disclosed that criminal hackers had compromised encrypted copies of some password vaults, as well as other personal information.

In February 2023, LastPass disclosed that the intrusion involved a highly complex, targeted attack against a DevOps engineer who was one of only four LastPass employees with access to the corporate vault.

“This was accomplished by targeting the DevOps engineer’s home computer and exploiting a vulnerable third-party media software package, which enabled remote code execution capability and allowed the threat actor to implant keylogger malware,” LastPass officials wrote. “The threat actor was able to capture the employee’s master password as it was entered, after the employee authenticated with MFA, and gain access to the DevOps engineer’s LastPass corporate vault.”

Dan Goodin at Ars Technica reported and then confirmed that the attackers exploited a known vulnerability in a Plex media server that the employee was running on his home network, and succeeded in installing malicious software that stole passwords and other authentication credentials. The vulnerability exploited by the intruders was patched back in 2020, but the employee never updated his Plex software.

As it happens, Plex announced its own data breach one day before LastPass disclosed its initial August intrusion. On August 24, 2022, Plex’s security team urged users to reset their passwords, saying an intruder had accessed customer emails, usernames and encrypted passwords.

OFFLINE ATTACKS

A basic functionality of LastPass is that it will pick and remember lengthy, complex passwords for each of your websites or online services. To automatically populate the appropriate credentials at any website going forward, you simply authenticate to LastPass using your master password.

LastPass has always emphasized that if you lose this master password, that’s too bad because they don’t store it and their encryption is so strong that even they can’t help you recover it.

But experts say all bets are off when cybercrooks can get their hands on the encrypted vault data itself — as opposed to having to interact with LastPass via its website. These so-called “offline” attacks allow the bad guys to conduct unlimited and unfettered “brute force” password cracking attempts against the encrypted data using powerful computers that can each try millions of password guesses per second.

“It does leave things vulnerable to brute force when the vaults are stolen en masse, especially if info about the vault HOLDER is available,” said Nicholas Weaver, a researcher at University of California, Berkeley’s International Computer Science Institute (ICSI) and lecturer at UC Davis. “So you just crunch and crunch and crunch with GPUs, with a priority list of vaults you target.”

How hard would it be for well-resourced criminals to crack the master passwords securing LastPass user vaults? Perhaps the best answer to this question comes from Wladimir Palant, a security researcher and the original developer behind the Adblock Plus browser plugin.

In a December 2022 blog post, Palant explained that the crackability of a LastPass master password depends largely on two things: The complexity of the master password, and the default settings for LastPass users, which appear to have varied quite a bit based on when those users began patronizing the service.

LastPass says that since 2018 it has required a twelve-character minimum for master passwords, which the company said “greatly minimizes the ability for successful brute force password guessing.”

But Palant said while LastPass indeed improved its master password defaults in 2018, it did not force all existing customers who had master passwords of lesser lengths to pick new credentials that would satisfy the 12-character minimum.

“If you are a LastPass customer, chances are that you are completely unaware of this requirement,” Palant wrote. “That’s because LastPass didn’t ask existing customers to change their master password. I had my test account since 2018, and even today I can log in with my eight-character password without any warnings or prompts to change it.”

Palant believes LastPass also failed to upgrade many older, original customers to more secure encryption protections that were offered to newer customers over the years. One important setting in LastPass is the number of “iterations,” or how many times your master password is run through the company’s encryption routines. The more iterations, the longer it takes an offline attacker to crack your master password.

Palant noted last year that for many older LastPass users, the initial default setting for iterations was anywhere from “1” to “500.” By 2013, new LastPass customers were given 5,000 iterations by default. In February 2018, LastPass changed the default to 100,100 iterations. And very recently, it upped that again to 600,000.

Palant said the 2018 change was in response to a security bug report he filed about some users having dangerously low iterations in their LastPass settings.

“Worse yet, for reasons that are beyond me, LastPass didn’t complete this migration,” Palant wrote. “My test account is still at 5,000 iterations, as are the accounts of many other users who checked their LastPass settings. LastPass would know how many users are affected, but they aren’t telling that. In fact, it’s painfully obvious that LastPass never bothered updating users’ security settings. Not when they changed the default from 1 to 500 iterations. Not when they changed it from 500 to 5,000. Only my persistence made them consider it for their latest change. And they still failed implementing it consistently.”

A chart on Palant’s blog post offers an idea of how increasing password iterations dramatically increases the costs and time needed by the attackers to crack someone’s master password. Palant said it would take a single GPU about a year to crack a password of average complexity with 500 iterations, and about 10 years to crack the same password run through 5,000 iterations.

However, these numbers radically come down when a determined adversary also has other large-scale computational assets at their disposal, such as a bitcoin mining operation that can coordinate the password-cracking activity across multiple powerful systems simultaneously.

Weaver said a password or passphrase with average complexity — such as “Correct Horse Battery Staple” is only secure against online attacks, and that its roughly 40 bits of randomness or “entropy” means a graphics card can blow through it in no time.

“An Nvidia 3090 can do roughly 4 million [password guesses] per second with 1000 iterations, but that would go down to 8 thousand per second with 500,000 iterations, which is why iteration count matters so much,” Weaver said. “So a combination of ‘not THAT strong of a password’ and ‘old vault’ and ‘low iteration count’ would make it theoretically crackable but real work, but the work is worth it given the targets.”

Reached by KrebsOnSecurity, Palant said he never received a response from LastPass about why the company apparently failed to migrate some number of customers to more secure account settings.

“I know exactly as much as everyone else,” Palant wrote in reply. “LastPass published some additional information in March. This finally answered the questions about the timeline of their breach – meaning which users are affected. It also made obvious that business customers are very much at risk here, Federated Login Services being highly compromised in this breach (LastPass downplaying as usual of course).”

Palant said upon logging into his LastPass account a few days ago, he found his master password was still set at 5,000 iterations.

INTERVIEW WITH A VICTIM

KrebsOnSecurity interviewed one of the victims tracked down by Monahan, a software engineer and startup founder who recently was robbed of approximately $3.4 million worth of different cryptocurrencies. The victim agreed to tell his story in exchange for anonymity because he is still trying to claw back his losses. We’ll refer to him here as “Connor” (not his real name).

Connor said he began using LastPass roughly a decade ago, and that he also stored the seed phrase for his primary cryptocurrency wallet inside of LastPass. Connor chose to protect his LastPass password vault with an eight character master password that included numbers and symbols (~50 bits of entropy).

“I thought at the time that the bigger risk was losing a piece of paper with my seed phrase on it,” Connor said. “I had it in a bank security deposit box before that, but then I started thinking, ‘Hey, the bank might close or burn down and I could lose my seed phrase.'”

Those seed phrases sat in his LastPass vault for years. Then, early on the morning of Sunday, Aug. 27, 2023, Connor was awoken by a service he’d set up to monitor his cryptocurrency addresses for any unusual activity: Someone was draining funds from his accounts, and fast.

Like other victims interviewed for this story, Connor didn’t suffer the usual indignities that typically presage a cryptocurrency robbery, such as account takeovers of his email inbox or mobile phone number.

Connor said he doesn’t know the number of iterations his master password was given originally, or what it was set at when the LastPass user vault data was stolen last year. But he said he recently logged into his LastPass account and the system forced him to upgrade to the new 600,000 iterations setting.

“Because I set up my LastPass account so early, I’m pretty sure I had whatever weak settings or iterations it originally had,” he said.

Connor said he’s kicking himself because he recently started the process of migrating his cryptocurrency to a new wallet protected by a new seed phrase. But he never finished that migration process. And then he got hacked.

“I’d set up a brand new wallet with new keys,” he said. “I had that ready to go two months ago, but have been procrastinating moving things to the new wallet.”

Connor has been exceedingly lucky in regaining access to some of his stolen millions in cryptocurrency. The Internet is swimming with con artists masquerading as legitimate cryptocurrency recovery experts. To make matters worse, because time is so critical in these crypto heists, many victims turn to the first quasi-believable expert who offers help.

Instead, several friends steered Connor to Flashbots.net, a cryptocurrency recovery firm that employs several custom techniques to help clients claw back stolen funds — particularly those on the Ethereum blockchain.

According to Connor, Flashbots helped rescue approximately $1.5 million worth of the $3.4 million in cryptocurrency value that was suddenly swept out of his account roughly a week ago. Lucky for him, Connor had some of his assets tied up in a type of digital loan that allowed him to borrow against his various cryptocurrency assets.

Without giving away too many details about how they clawed back the funds, here’s a high level summary: When the crooks who stole Connor’s seed phrase sought to extract value from these loans, they were borrowing the maximum amount of credit that he hadn’t already used. But Connor said that left open an avenue for some of that value to be recaptured, basically by repaying the loan in many small, rapid chunks.

WHAT SHOULD LASTPASS USERS DO?

According to MetaMask’s Monahan, users who stored any important passwords with LastPass — particularly those related to cryptocurrency accounts — should change those credentials immediately, and migrate any crypto holdings to new offline hardware wallets.

“Really the ONLY thing you need to read is this,” Monahan pleaded to her 70,000 followers on Twitter/X: “PLEASE DON’T KEEP ALL YOUR ASSETS IN A SINGLE KEY OR SECRET PHRASE FOR YEARS. THE END. Split up your assets. Get a hw [hardware] wallet. Migrate. Now.”

If you also had passwords tied to banking or retirement accounts, or even just important email accounts — now would be a good time to change those credentials as well.

I’ve never been comfortable recommending password managers, because I’ve never seriously used them myself. Something about putting all your eggs in one basket. Heck, I’m so old-fashioned that most of my important passwords are written down and tucked away in safe places.

But I recognize this antiquated approach to password management is not for everyone. Connor says he now uses 1Password, a competing password manager that recently earned the best overall marks from Wired and The New York Times.

1Password says that three things are needed to decrypt your information: The encrypted data itself, your account password, and your Secret Key. Only you know your account password, and your Secret Key is generated locally during setup.

“The two are combined on-device to encrypt your vault data and are never sent to 1Password,” explains a 1Password blog post ‘What If 1Password Gets Hacked?‘ “Only the encrypted vault data lives on our servers, so neither 1Password nor an attacker who somehow manages to guess or steal your account password would be able to access your vaults – or what’s inside them.

Weaver said that Secret Key adds an extra level of randomness to all user master passwords that LastPass didn’t have.

“With LastPass, the idea is the user’s password vault is encrypted with a cryptographic hash (H) of the user’s passphrase,” Weaver said. “The problem is a hash of the user’s passphrase is remarkably weak on older LastPass vaults with master passwords that do not have many iterations. 1Password uses H(random-key||password) to generate the password, and it is why you have the QR code business when adding a new device.”

Weaver said LastPass deserves blame for not having upgraded iteration counts for all users a long time ago, and called the latest forced upgrades “a stunning indictment of the negligence on the part of LastPass.”

“That they never even notified all those with iteration counts of less than 100,000 — who are really vulnerable to brute force even with 8-character random passwords or ‘correct horse battery staple’ type passphrases — is outright negligence,” Weaver said. “I would personally advocate that nobody ever uses LastPass again: Not because they were hacked. Not because they had an architecture (unlike 1Password) that makes such hacking a problem. But because of their consistent refusal to address how they screwed up and take proactive efforts to protect their customers.”

Bax and Monahan both acknowledged that their research alone can probably never conclusively tie dozens of high-dollar crypto heists over the past year to the LastPass breach. But Bax says at this point he doesn’t see any other possible explanation.

“Some might say it’s dangerous to assert a strong connection here, but I’d say it’s dangerous to assert there isn’t one,” he said. “I was arguing with my fiance about this last night. She’s waiting for LastPass to tell her to change everything. Meanwhile, I’m telling her to do it now.”

Barracuda Urges Replacing — Not Patching — Its Email Security Gateways

It’s not often that a zero-day vulnerability causes a network security vendor to urge customers to physically remove and decommission an entire line of affected hardware — as opposed to just applying software updates. But experts say that is exactly what transpired this week with Barracuda Networks, as the company struggled to combat a sprawling malware threat which appears to have undermined its email security appliances in such a fundamental way that they can no longer be safely updated with software fixes.

Campbell, Calif. based Barracuda said it hired incident response firm Mandiant on May 18 after receiving reports about unusual traffic originating from its Email Security Gateway (ESG) devices, which are designed to sit at the edge of an organization’s network and scan all incoming and outgoing email for malware.

On May 19, Barracuda identified that the malicious traffic was taking advantage of a previously unknown vulnerability in its ESG appliances, and on May 20 the company pushed a patch for the flaw to all affected appliances (CVE-2023-2868).

In its security advisory, Barracuda said the vulnerability existed in the Barracuda software component responsible for screening attachments for malware. More alarmingly, the company said it appears attackers first started exploiting the flaw in October 2022.

But on June 6, Barracuda suddenly began urging its ESG customers to wholesale rip out and replace — not patch — affected appliances.

“Impacted ESG appliances must be immediately replaced regardless of patch version level,” the company’s advisory warned. “Barracuda’s recommendation at this time is full replacement of the impacted ESG.”

In a statement, Barracuda said it will be providing the replacement product to impacted customers at no cost, and that not all ESG appliances were compromised.

“No other Barracuda product, including our SaaS email solutions, were impacted by this vulnerability,” the company said. “If an ESG appliance is displaying a notification in the User Interface, the ESG appliance had indicators of compromise. If no notification is displayed, we have no reason to believe that the appliance has been compromised at this time.”

Nevertheless, the statement says that “out of an abundance of caution and in furtherance of our containment strategy, we recommend impacted customers replace their compromised appliance.”

“As of June 8, 2023, approximately 5% of active ESG appliances worldwide have shown any evidence of known indicators of compromise due to the vulnerability,” the statement continues. “Despite deployment of additional patches based on known IOCs, we continue to see evidence of ongoing malware activity on a subset of the compromised appliances. Therefore, we would like customers to replace any compromised appliance with a new unaffected device.”

Rapid7‘s Caitlin Condon called this remarkable turn of events “fairly stunning,” and said there appear to be roughly 11,000 vulnerable ESG devices still connected to the Internet worldwide.

“The pivot from patch to total replacement of affected devices is fairly stunning and implies the malware the threat actors deployed somehow achieves persistence at a low enough level that even wiping the device wouldn’t eradicate attacker access,” Condon wrote.

Barracuda said the malware was identified on a subset of appliances that allowed the attackers persistent backdoor access to the devices, and that evidence of data exfiltration was identified on some systems.

Rapid7 said it has seen no evidence that attackers are using the flaw to move laterally within victim networks. But that may be small consolation for Barracuda customers now coming to terms with the notion that foreign cyberspies probably have been hoovering up all their email for months.

Nicholas Weaver, a researcher at University of California, Berkeley’s International Computer Science Institute (ICSI), said it is likely that the malware was able to corrupt the underlying firmware that powers the ESG devices in some irreparable way.

“One of the goals of malware is to be hard to remove, and this suggests the malware compromised the firmware itself to make it really hard to remove and really stealthy,” Weaver said. “That’s not a ransomware actor, that’s a state actor. Why? Because a ransomware actor doesn’t care about that level of access. They don’t need it. If they’re going for data extortion, it’s more like a smash-and-grab. If they’re going for data ransoming, they’re encrypting the data itself — not the machines.”

In addition to replacing devices, Barracuda says ESG customers should also rotate any credentials connected to the appliance(s), and check for signs of compromise dating back to at least October 2022 using the network and endpoint indicators the company has released publicly.

Update, June 9, 11:55 a.m. ET: Barracuda has issued an updated statement about the incident, portions of which are now excerpted above.

Hackers Claim They Breached T-Mobile More Than 100 Times in 2022

Three different cybercriminal groups claimed access to internal networks at communications giant T-Mobile in more than 100 separate incidents throughout 2022, new data suggests. In each case, the goal of the attackers was the same: Phish T-Mobile employees for access to internal company tools, and then convert that access into a cybercrime service that could be hired to divert any T-Mobile user’s text messages and phone calls to another device.

The conclusions above are based on an extensive analysis of Telegram chat logs from three distinct cybercrime groups or actors that have been identified by security researchers as particularly active in and effective at “SIM-swapping,” which involves temporarily seizing control over a target’s mobile phone number.

Countless websites and online services use SMS text messages for both password resets and multi-factor authentication. This means that stealing someone’s phone number often can let cybercriminals hijack the target’s entire digital life in short order — including access to any financial, email and social media accounts tied to that phone number.

All three SIM-swapping entities that were tracked for this story remain active in 2023, and they all conduct business in open channels on the instant messaging platform Telegram. KrebsOnSecurity is not naming those channels or groups here because they will simply migrate to more private servers if exposed publicly, and for now those servers remain a useful source of intelligence about their activities.

Each advertises their claimed access to T-Mobile systems in a similar way. At a minimum, every SIM-swapping opportunity is announced with a brief “Tmobile up!” or “Tmo up!” message to channel participants. Other information in the announcements includes the price for a single SIM-swap request, and the handle of the person who takes the payment and information about the targeted subscriber.

The information required from the customer of the SIM-swapping service includes the target’s phone number, and the serial number tied to the new SIM card that will be used to receive text messages and phone calls from the hijacked phone number.

Initially, the goal of this project was to count how many times each entity claimed access to T-Mobile throughout 2022, by cataloging the various “Tmo up!” posts from each day and working backwards from Dec. 31, 2022.

But by the time we got to claims made in the middle of May 2022, completing the rest of the year’s timeline seemed unnecessary. The tally shows that in the last seven-and-a-half months of 2022, these groups collectively made SIM-swapping claims against T-Mobile on 104 separate days — often with multiple groups claiming access on the same days.

KrebsOnSecurity shared a large amount of data gathered for this story with T-Mobile. The company declined to confirm or deny any of these claimed intrusions. But in a written statement, T-Mobile said this type of activity affects the entire wireless industry.

“And we are constantly working to fight against it,” the statement reads. “We have continued to drive enhancements that further protect against unauthorized access, including enhancing multi-factor authentication controls, hardening environments, limiting access to data, apps or services, and more. We are also focused on gathering threat intelligence data, like what you have shared, to help further strengthen these ongoing efforts.”

TMO UP!

While it is true that each of these cybercriminal actors periodically offer SIM-swapping services for other mobile phone providers — including AT&T, Verizon and smaller carriers — those solicitations appear far less frequently in these group chats than T-Mobile swap offers. And when those offers do materialize, they are considerably more expensive.

The prices advertised for a SIM-swap against T-Mobile customers in the latter half of 2022 ranged between USD $1,000 and $1,500, while SIM-swaps offered against AT&T and Verizon customers often cost well more than twice that amount.

To be clear, KrebsOnSecurity is not aware of specific SIM-swapping incidents tied to any of these breach claims. However, the vast majority of advertisements for SIM-swapping claims against T-Mobile tracked in this story had two things in common that set them apart from random SIM-swapping ads on Telegram.

First, they included an offer to use a mutually trusted “middleman” or escrow provider for the transaction (to protect either party from getting scammed). More importantly, the cybercriminal handles that were posting ads for SIM-swapping opportunities from these groups generally did so on a daily or near-daily basis — often teasing their upcoming swap events in the hours before posting a “Tmo up!” message announcement.

In other words, if the crooks offering these SIM-swapping services were ripping off their customers or claiming to have access that they didn’t, this would be almost immediately obvious from the responses of the more seasoned and serious cybercriminals in the same chat channel.

There are plenty of people on Telegram claiming to have SIM-swap access at major telecommunications firms, but a great many such offers are simply four-figure scams, and any pretenders on this front are soon identified and banned (if not worse).

One of the groups that reliably posted “Tmo up!” messages to announce SIM-swap availability against T-Mobile customers also reliably posted “Tmo down!” follow-up messages announcing exactly when their claimed access to T-Mobile employee tools was discovered and revoked by the mobile giant.

A review of the timestamps associated with this group’s incessant “Tmo up” and “Tmo down” posts indicates that while their claimed access to employee tools usually lasted less than an hour, in some cases that access apparently went undiscovered for several hours or even days.

TMO TOOLS

How could these SIM-swapping groups be gaining access to T-Mobile’s network as frequently as they claim? Peppered throughout the daily chit-chat on their Telegram channels are solicitations for people urgently needed to serve as “callers,” or those who can be hired to social engineer employees over the phone into navigating to a phishing website and entering their employee credentials.

Allison Nixon is chief research officer for the New York City-based cybersecurity firm Unit 221B. Nixon said these SIM-swapping groups will typically call employees on their mobile devices, pretend to be someone from the company’s IT department, and then try to get the person on the other end of the line to visit a phishing website that mimics the company’s employee login page.

Nixon argues that many people in the security community tend to discount the threat from voice phishing attacks as somehow “low tech” and “low probability” threats.

“I see it as not low-tech at all, because there are a lot of moving parts to phishing these days,” Nixon said. “You have the caller who has the employee on the line, and the person operating the phish kit who needs to spin it up and down fast enough so that it doesn’t get flagged by security companies. Then they have to get the employee on that phishing site and steal their credentials.”

In addition, she said, often there will be yet another co-conspirator whose job it is to use the stolen credentials and log into employee tools. That person may also need to figure out how to make their device pass “posture checks,” a form of device authentication that some companies use to verify that each login is coming only from employer-issued phones or laptops.

For aspiring criminals with little experience in scam calling, there are plenty of sample call transcripts available on these Telegram chat channels that walk one through how to impersonate an IT technician at the targeted company — and how to respond to pushback or skepticism from the employee. Here’s a snippet from one such tutorial that appeared recently in one of the SIM-swapping channels:

“Hello this is James calling from Metro IT department, how’s your day today?”

(yea im doing good, how r u)

i’m doing great, thank you for asking

i’m calling in regards to a ticket we got last week from you guys, saying you guys were having issues with the network connectivity which also interfered with [Microsoft] Edge, not letting you sign in or disconnecting you randomly. We haven’t received any updates to this ticket ever since it was created so that’s why I’m calling in just to see if there’s still an issue or not….”

TMO DOWN!

The TMO UP data referenced above, combined with comments from the SIM-swappers themselves, indicate that while many of their claimed accesses to T-Mobile tools in the middle of 2022 lasted hours on end, both the frequency and duration of these events began to steadily decrease as the year wore on.

T-Mobile declined to discuss what it may have done to combat these apparent intrusions last year. However, one of the groups began to complain loudly in late October 2022 that T-Mobile must have been doing something that was causing their phished access to employee tools to die very soon after they obtained it.

One group even remarked that they suspected T-Mobile’s security team had begun monitoring their chats.

Indeed, the timestamps associated with one group’s TMO UP/TMO DOWN notices show that their claimed access was often limited to less than 15 minutes throughout November and December of 2022.

Whatever the reason, the calendar graphic above clearly shows that the frequency of claimed access to T-Mobile decreased significantly across all three SIM-swapping groups in the waning weeks of 2022.

SECURITY KEYS

T-Mobile US reported revenues of nearly $80 billion last year. It currently employs more than 71,000 people in the United States, any one of whom can be a target for these phishers.

T-Mobile declined to answer questions about what it may be doing to beef up employee authentication. But Nicholas Weaver, a researcher and lecturer at University of California, Berkeley’s International Computer Science Institute, said T-Mobile and all the major wireless providers should be requiring employees to use physical security keys for that second factor when logging into company resources.

“These breaches should not happen,” Weaver said. “Because T-Mobile should have long ago issued all employees security keys and switched to security keys for the second factor. And because security keys provably block this style of attack.”

The most commonly used security keys are inexpensive USB-based devices. A security key implements a form of multi-factor authentication known as Universal 2nd Factor (U2F), which allows the user to complete the login process simply by inserting the USB key and pressing a button on the device. The key works without the need for any special software drivers.

The allure of U2F devices for multi-factor authentication is that even if an employee who has enrolled a security key for authentication tries to log in at an impostor site, the company’s systems simply refuse to request the security key if the user isn’t on their employer’s legitimate website, and the login attempt fails. Thus, the second factor cannot be phished, either over the phone or Internet.

THE ROLE OF MINORS IN SIM-SWAPPING

Nixon said one confounding aspect of SIM-swapping is that these criminal groups tend to recruit teenagers to do their dirty work.

“A huge reason this problem has been allowed to spiral out of control is because children play such a prominent role in this form of breach,” Nixon said.

Nixon said SIM-swapping groups often advertise low-level jobs on places like Roblox and Minecraft, online games that are extremely popular with young adolescent males.

“Statistically speaking, that kind of recruiting is going to produce a lot of people who are underage,” she said. “They recruit children because they’re naive, you can get more out of them, and they have legal protections that other people over 18 don’t have.”

For example, she said, even when underage SIM-swappers are arrested, the offenders tend to go right back to committing the same crimes as soon as they’re released.

In January 2023, T-Mobile disclosed that a “bad actor” stole records on roughly 37 million current customers, including their name, billing address, email, phone number, date of birth, and T-Mobile account number.

In August 2021, T-Mobile acknowledged that hackers made off with the names, dates of birth, Social Security numbers and driver’s license/ID information on more than 40 million current, former or prospective customers who applied for credit with the company. That breach came to light after a hacker began selling the records on a cybercrime forum.

In the shadow of such mega-breaches, any damage from the continuous attacks by these SIM-swapping groups can seem insignificant by comparison. But Nixon says it’s a mistake to dismiss SIM-swapping as a low volume problem.

“Logistically, you may only be able to get a few dozen or a hundred SIM-swaps in a day, but you can pick any customer you want across their entire customer base,” she said. “Just because a targeted account takeover is low volume doesn’t mean it’s low risk. These guys have crews that go and identify people who are high net worth individuals and who have a lot to lose.”

Nixon said another aspect of SIM-swapping that causes cybersecurity defenders to dismiss the threat from these groups is the perception that they are full of low-skilled “script kiddies,” a derisive term used to describe novice hackers who rely mainly on point-and-click hacking tools.

“They underestimate these actors and say this person isn’t technically sophisticated,” she said. “But if you’re rolling around in millions worth of stolen crypto currency, you can buy that sophistication. I know for a fact some of these compromises were at the hands of these ‘script kiddies,’ but they’re not ripping off other people’s scripts so much as hiring people to make scripts for them. And they don’t care what gets the job done, as long as they get to steal the money.”

Battle with Bots Prompts Mass Purge of Amazon, Apple Employee Accounts on LinkedIn

On October 10, 2022, there were 576,562 LinkedIn accounts that listed their current employer as Apple Inc. The next day, half of those profiles no longer existed. A similarly dramatic drop in the number of LinkedIn profiles claiming employment at Amazon comes as LinkedIn is struggling to combat a significant uptick in the creation of fake employee accounts that pair AI-generated profile photos with text lifted from legitimate users.

Jay Pinho is a developer who is working on a product that tracks company data, including hiring. Pinho has been using LinkedIn to monitor daily employee headcounts at several dozen large organizations, and last week he noticed that two of them had far fewer people claiming to work for them than they did just 24 hours previously.

Pinho’s screenshot below shows the daily count of employees as displayed on Amazon’s LinkedIn homepage. Pinho said his scraper shows that the number of LinkedIn profiles claiming current roles at Amazon fell from roughly 1.25 million to 838,601 in just one day, a 33 percent drop:

As stated above, the number of LinkedIn profiles that claimed to work at Apple fell by approximately 50 percent on Oct. 10, according to Pinho’s analysis:

Neither Amazon or Apple responded to requests for comment. LinkedIn declined to answer questions about the account purges, saying only that the company is constantly working to keep the platform free of fake accounts. In June, LinkedIn acknowledged it was seeing a rise in fraudulent activity happening on the platform.

KrebsOnSecurity hired Menlo Park, Calif.-based SignalHire to check Pinho’s numbers. SignalHire keeps track of active and former profiles on LinkedIn, and during the Oct 9-11 timeframe SignalHire said it saw somewhat smaller but still unprecedented drops in active profiles tied to Amazon and Apple.

“The drop in the percentage of 7-10 percent [of all profiles], as it happened [during] this time, is not something that happened before,” SignalHire’s Anastacia Brown told KrebsOnSecurity.

Brown said the normal daily variation in profile numbers for these companies is plus or minus one percent.

“That’s definitely the first huge drop that happened throughout the time we’ve collected the profiles,” she said.

In late September 2022, KrebsOnSecurity warned about the proliferation of fake LinkedIn profiles for Chief Information Security Officer (CISO) roles at some of the world’s largest corporations. A follow-up story on Oct. 5 showed how the phony profile problem has affected virtually all executive roles at corporations, and how these fake profiles are creating an identity crisis for the businesses networking site and the companies that rely on it to hire and screen prospective employees.

A day after that second story ran, KrebsOnSecurity heard from a recruiter who noticed the number of LinkedIn profiles that claimed virtually any role in network security had dropped seven percent overnight. LinkedIn declined to comment about that earlier account purge, saying only that, “We’re constantly working at taking down fake accounts.”

It’s unclear whether LinkedIn is responsible for this latest account purge, or if individually affected companies are starting to take action on their own. The timing, however, argues for the former, as the account purges for Apple and Amazon employees tracked by Pinho appeared to happen within the same 24 hour period.

It’s also unclear who or what is behind the recent proliferation of fake executive profiles on LinkedIn. Cybersecurity firm Mandiant (recently acquired by Google) told Bloomberg that hackers working for the North Korean government have been copying resumes and profiles from leading job listing platforms LinkedIn and Indeed, as part of an elaborate scheme to land jobs at cryptocurrency firms.

On this point, Pinho said he noticed an account purge in early September that targeted fake profiles tied to jobs at cryptocurrency exchange Binance. Up until Sept. 3, there were 7,846 profiles claiming current executive roles at Binance. The next day, that number stood at 6,102, a 23 percent drop (by some accounts that 6,102 head count is still wildly inflated).

Fake profiles also may be tied to so-called “pig butchering” scams, wherein people are lured by flirtatious strangers online into investing in cryptocurrency trading platforms that eventually seize any funds when victims try to cash out.

In addition, identity thieves have been known to masquerade on LinkedIn as job recruiters, collecting personal and financial information from people who fall for employment scams.

Nicholas Weaver, a researcher for the International Computer Science Institute at University of California, Berkeley, suggested another explanation for the recent glut of phony LinkedIn profiles: Someone may be setting up a mass network of accounts in order to more fully scrape profile information from the entire platform.

“Even with just a standard LinkedIn account, there’s a pretty good amount of profile information just in the default two-hop networks,” Weaver said. “We don’t know the purpose of these bots, but we know creating bots isn’t free and creating hundreds of thousands of bots would require a lot of resources.”

In response to last week’s story about the explosion of phony accounts on LinkedIn, the company said it was exploring new ways to protect members, such as expanding email domain verification. Under such a scheme, LinkedIn users would be able to publicly attest that their profile is accurate by verifying that they can respond to email at the domain associated with their current employer.

LinkedIn claims that its security systems detect and block approximately 96 percent of fake accounts. And despite the recent purges, LinkedIn may be telling the truth, Weaver said.

“There’s no way you can test for that,” he said. “Because technically, it may be that there were actually 100 million bots trying to sign up at LinkedIn as employees at Amazon.”

Weaver said the apparent mass account purge at LinkedIn underscores the size of the bot problem, and could present a “real and material change” for LinkedIn.

“It may mean the statistics they’ve been reporting about usage and active accounts are off by quite a bit,” Weaver said.

Experian, You Have Some Explaining to Do

Twice in the past month KrebsOnSecurity has heard from readers who had their accounts at big-three credit bureau Experian hacked and updated with a new email address that wasn’t theirs. In both cases the readers used password managers to select strong, unique passwords for their Experian accounts. Research suggests identity thieves were able to hijack the accounts simply by signing up for new accounts at Experian using the victim’s personal information and a different email address.

John Turner is a software engineer based in Salt Lake City. Turner said he created the account at Experian in 2020 to place a security freeze on his credit file, and that he used a password manager to select and store a strong, unique password for his Experian account.

Turner said that in early June 2022 he received an email from Experian saying the email address on his account had been changed. Experian’s password reset process was useless at that point because any password reset links would be sent to the new (impostor’s) email address.

An Experian support person Turner reached via phone after a lengthy hold time asked for his Social Security Number (SSN) and date of birth, as well as his account PIN and answers to his secret questions. But the PIN and secret questions had already been changed by whoever re-signed up as him at Experian.

“I was able to answer the credit report questions successfully, which authenticated me to their system,” Turner said. “At that point, the representative read me the current stored security questions and PIN, and they were definitely not things I would have used.”

Turner said he was able to regain control over his Experian account by creating a new account. But now he’s wondering what else he could do to prevent another account compromise.

“The most frustrating part of this whole thing is that I received multiple ‘here’s your login information’ emails later that I attributed to the original attackers coming back and attempting to use the ‘forgot email/username’ flow, likely using my SSN and DOB, but it didn’t go to their email that they were expecting,” Turner said. “Given that Experian doesn’t support two-factor authentication of any kind — and that I don’t know how they were able to get access to my account in the first place — I’ve felt very helpless ever since.”

Arthur Rishi is a musician and co-executive director of the Boston Landmarks Orchestra. Rishi said he recently discovered his Experian account had been hijacked after receiving an alert from his credit monitoring service (not Experian’s) that someone had tried to open an account in his name at JPMorgan Chase.

Rishi said the alert surprised him because his credit file at Experian was frozen at the time, and Experian did not notify him about any activity on his account. Rishi said Chase agreed to cancel the unauthorized account application, and even rescinded its credit inquiry (each credit pull can ding your credit score slightly).

But he never could get anyone from Experian’s support to answer the phone, despite spending what seemed like eternity trying to progress through the company’s phone-based system. That’s when Rishi decided to see if he could create a new account for himself at Experian.

“I was able to open a new account at Experian starting from scratch, using my SSN, date of birth and answering some really basic questions, like what kind of car did you take out a loan for, or what city did you used to live in,’ Rishi said.

Upon completing the sign-up, Rishi noticed that his credit was unfrozen.

Like Turner, Rishi is now worried that identity thieves will just hijack his Experian account once more, and that there is nothing he can do to prevent such a scenario. For now, Rishi has decided to pay Experian $25.99 a month to more closely monitor his account for suspicious activity. Even using the paid Experian service, there were no additional multi-factor authentication options available, although he said Experian did send a one-time code to his phone via SMS recently when he logged on.

“Experian now sometimes does require MFA for me if I use a new browser or have my VPN on,” Rishi said, but he’s not sure if Experian’s free service would have operated differently.

“I get so angry when I think about all this,” he said. “I have no confidence this won’t happen again.”

In a written statement, Experian suggested that what happened to Rishi and Turner was not a normal occurrence, and that its security and identity verification practices extend beyond what is visible to the user.

“We believe these are isolated incidents of fraud using stolen consumer information,” Experian’s statement reads. “Specific to your question, once an Experian account is created, if someone attempts to create a second Experian account, our systems will notify the original email on file.”

“We go beyond reliance on personally identifiable information (PII) or a consumer’s ability to answer knowledge-based authentication questions to access our systems,” the statement continues. “We do not disclose additional processes for obvious security reasons; however, our data and analytical capabilities verify identity elements across multiple data sources and are not visible to the consumer. This is designed to create a more positive experience for our consumers and to provide additional layers of protection. We take consumer privacy and security seriously, and we continually review our security processes to guard against constant and evolving threats posed by fraudsters.”

ANALYSIS

KrebsOnSecurity sought to replicate Turner and Rishi’s experience — to see if Experian would allow me to re-create my account using my personal information but a different email address. The experiment was done from a different computer and Internet address than the one that created the original account years ago.

After providing my Social Security Number (SSN), date of birth, and answering several multiple choice questions whose answers are derived almost entirely from public records, Experian promptly changed the email address associated with my credit file. It did so without first confirming that new email address could respond to messages, or that the previous email address approved the change.

Experian’s system then sent an automated message to the original email address on file, saying the account’s email address had been changed. The only recourse Experian offered in the alert was to sign in, or send an email to an Experian inbox that replies with the message, “this email address is no longer monitored.”

After that, Experian prompted me to select new secret questions and answers, as well as a new account PIN — effectively erasing the account’s previously chosen PIN and recovery questions. Once I’d changed the PIN and security questions, Experian’s site helpfully reminded me that I have a security freeze on file, and would I like to remove or temporarily lift the security freeze?

To be clear, Experian does have a business unit that sells one-time password services to businesses. While Experian’s system did ask for a mobile number when I signed up a second time, at no time did that number receive a notification from Experian. Also, I could see no option in my account to enable multi-factor authentication for all logins.

How does Experian differ from the practices of Equifax and TransUnion, the other two big consumer credit reporting bureaus? When KrebsOnSecurity tried to re-create an existing account at TransUnion using my Social Security number, TransUnion rejected the application, noting that I already had an account and prompting me to proceed through its lost password flow. The company also appears to send an email to the address on file asking to validate account changes.

Likewise, trying to recreate an existing account at Equifax using personal information tied to my existing account prompts Equifax’s systems to report that I already have an account, and to use their password reset process (which involves sending a verification email to the address on file).

KrebsOnSecurity has long urged readers in the United States to place a security freeze on their files with the three major credit bureaus. With a freeze in place, potential creditors can’t pull your credit file, which makes it very unlikely anyone will be granted new lines of credit in your name. I’ve also advised readers to plant their flag at the three major bureaus, to prevent identity thieves from creating an account for you and assuming control over your identity.

The experiences of Rishi, Turner and this author suggest Experian’s practices currently undermine both of those proactive security measures. Even so, having an active account at Experian may be the only way you find out when crooks have assumed your identity. Because at least then you should receive an email from Experian saying they gave your identity to someone else.

In April 2021, KrebsOnSecurity revealed how identity thieves were exploiting lax authentication on Experian’s PIN retrieval page to unfreeze consumer credit files. In those cases, Experian failed to send any notice via email when a freeze PIN was retrieved, nor did it require the PIN to be sent to an email address already associated with the consumer’s account.

A few days after that April 2021 story, KrebsOnSecurity broke the news that an Experian API was exposing the credit scores of most Americans.

Emory Roan, policy counsel for the Privacy Rights Clearinghouse, said Experian not offering multi-factor authentication for consumer accounts is inexcusable in 2022.

“They compound the problem by gating the recovery process with information that’s likely available or inferable from third party data brokers, or that could have been exposed in previous data breaches,” Roan said. “Experian is one of the largest Consumer Reporting Agencies in the country, trusted as one of the few essential players in a credit system Americans are forced to be part of. For them to not offer consumers some form of (free) MFA is baffling and reflects extremely poorly on Experian.”

Nicholas Weaver, a researcher for the International Computer Science Institute at University of California, Berkeley, said Experian has no real incentive to do things right on the consumer side of its business. That is, he said, unless Experian’s customers — banks and other lenders — choose to vote with their feet because too many people with frozen credit files are having to deal with unauthorized applications for new credit.

“The actual customers of the credit service don’t realize how much worse Experian is, and this isn’t the first time Experian has screwed up horribly,” Weaver said. “Experian is part of a triopoly, and I’m sure this is costing their actual customers money, because if you have a credit freeze that gets lifted and somebody loans against it, it’s the lender who eats that fraud cost.”

And unlike consumers, he said, lenders do have a choice in which of the triopoly handles their credit checks.

“I do think it’s important to point out that their real customers do have a choice, and they should switch to TransUnion and Equifax,” he added.

More greatest hits from Experian:

2017: Experian Site Can Give Anyone Your Credit Freeze PIN
2015: Experian Breach Affects 15 Million Customers
2015: Experian Breach Tied to NY-NJ ID Theft Ring
2015: At Experian, Security Attrition Amid Acquisitions
2015: Experian Hit With Class Action Over ID Theft Service
2014: Experian Lapse Allowed ID Theft Service Access to 200 Million Consumer Records
2013: Experian Sold Consumer Data to ID Theft Service

Update, 10:32 a.m.: Updated the story to clarify that while Experian does sometimes ask users to enter a one-time code sent via SMS to the number on file, there does not appear to be any option to enable this on all logins.

DEA Investigating Breach of Law Enforcement Data Portal

The U.S. Drug Enforcement Administration (DEA) says it is investigating reports that hackers gained unauthorized access to an agency portal that taps into 16 different federal law enforcement databases. KrebsOnSecurity has learned the alleged compromise is tied to a cybercrime and online harassment community that routinely impersonates police and government officials to harvest personal information on their targets.

On May 8, KrebsOnSecurity received a tip that hackers obtained a username and password for an authorized user of esp.usdoj.gov, which is the Law Enforcement Inquiry and Alerts (LEIA) system managed by the DEA.

KrebsOnSecurity shared information about the allegedly hijacked account with the DEA, the Federal Bureau of Investigation (FBI), and the Department of Justice, which houses both agencies. The DEA declined to comment on the validity of the claims, issuing only a brief statement in response.

“DEA takes cyber security and information of intrusions seriously and investigates all such reports to the fullest extent,” the agency said in a statement shared via email.

According to this page at the Justice Department website, LEIA “provides federated search capabilities for both EPIC and external database repositories,” including data classified as “law enforcement sensitive” and “mission sensitive” to the DEA.

A document published by the Obama administration in May 2016 (PDF) says the DEA’s El Paso Intelligence Center (EPIC) systems in Texas are available for use by federal, state, local and tribal law enforcement, as well as the Department of Defense and intelligence community.

EPIC and LEIA also have access to the DEA’s National Seizure System (NSS), which the DEA uses to identify property thought to have been purchased with the proceeds of criminal activity (think fancy cars, boats and homes seized from drug kingpins).

“The EPIC System Portal (ESP) enables vetted users to remotely and securely share intelligence, access the National Seizure System, conduct data analytics, and obtain information in support of criminal investigations or law enforcement operations,” the 2016 White House document reads. “Law Enforcement Inquiry and Alerts (LEIA) allows for a federated search of 16 Federal law enforcement databases.”

The screenshots shared with this author indicate the hackers could use EPIC to look up a variety of records, including those for motor vehicles, boats, firearms, aircraft, and even drones.

Claims about the purloined DEA access were shared with this author by “KT,” the current administrator of the Doxbin — a highly toxic online community that provides a forum for digging up personal information on people and posting it publicly.

As KrebsOnSecurity reported earlier this year, the previous owner of the Doxbin has been identified as the leader of LAPSUS$, a data extortion group that hacked into some of the world’s largest tech companies this year — including Microsoft, NVIDIA, Okta, Samsung and T-Mobile.

That reporting also showed how the core members of LAPSUS$ were involved in selling a service offering fraudulent Emergency Data Requests (EDRs), wherein the hackers use compromised police and government email accounts to file warrantless data requests with social media firms, mobile telephony providers and other technology firms, attesting that the information being requested can’t wait for a warrant because it relates to an urgent matter of life and death.

From the standpoint of individuals involved in filing these phony EDRs, access to databases and user accounts within the Department of Justice would be a major coup. But the data in EPIC would probably be far more valuable to organized crime rings or drug cartels, said Nicholas Weaver, a researcher for the International Computer Science Institute at University of California, Berkeley.

Weaver said it’s clear from the screenshots shared by the hackers that they could use their access not only to view sensitive information, but also submit false records to law enforcement and intelligence agency databases.

“I don’t think these [people] realize what they got, how much money the cartels would pay for access to this,” Weaver said. “Especially because as a cartel you don’t search for yourself you search for your enemies, so that even if it’s discovered there is no loss to you of putting things ONTO the DEA’s radar.”

ANALYSIS

The login page for esp.usdoj.gov (above) suggests that authorized users can access the site using a “Personal Identity Verification” or PIV card, which is a fairly strong form of authentication used government-wide to control access to federal facilities and information systems at each user’s appropriate security level.

However, the EPIC portal also appears to accept just a username and password, which would seem to radically diminish the security value of requiring users to present (or prove possession of) an authorized PIV card. Indeed, KT said the hacker who obtained this illicit access was able to log in using the stolen credentials alone, and that at no time did the portal prompt for a second authentication factor.

It’s not clear why there are still sensitive government databases being protected by nothing more than a username and password, but I’m willing to bet big money that this DEA portal is not only offender here. The DEA portal esp.usdoj.gov is listed on Page 87 of a Justice Department “data inventory,” which catalogs all of the data repositories that correspond to DOJ agencies.

There are 3,330 results. Granted, only some of those results are login portals, but that’s just within the Department of Justice.

If we assume for the moment that state-sponsored foreign hacking groups can gain access to sensitive government intelligence in the same way as teenage hacker groups like LAPSUS$, then it is long past time for the U.S. federal government to perform a top-to-bottom review of authentication requirements tied to any government portals that traffic in sensitive or privileged information.

I’ll say it because it needs to be said: The United States government is in urgent need of leadership on cybersecurity at the executive branch level — preferably someone who has the authority and political will to eventually disconnect any federal government agency data portals that fail to enforce strong, multi-factor authentication.

I realize this may be far more complex than it sounds, particularly when it comes to authenticating law enforcement personnel who access these systems without the benefit of a PIV card or government-issued device (state and local authorities, for example). It’s not going to be as simple as just turning on multi-factor authentication for every user, thanks in part to a broad diversity of technologies being used across the law enforcement landscape.

But when hackers can plunder 16 law enforcement databases, arbitrarily send out law enforcement alerts for specific people or vehicles, or potentially disrupt ongoing law enforcement operations — all because someone stole, found or bought a username and password — it’s time for drastic measures.

Your Phone May Soon Replace Many of Your Passwords

Apple, Google and Microsoft announced this week they will soon support an approach to authentication that avoids passwords altogether, and instead requires users to merely unlock their smartphones to sign in to websites or online services. Experts say the changes should help defeat many types of phishing attacks and ease the overall password burden on Internet users, but caution that a true passwordless future may still be years away for most websites.

The tech giants are part of an industry-led effort to replace passwords, which are easily forgotten, frequently stolen by malware and phishing schemes, or leaked and sold online in the wake of corporate data breaches.

Apple, Google and Microsoft are some of the more active contributors to a passwordless sign-in standard crafted by the FIDO (“Fast Identity Online”) Alliance and the World Wide Web Consortium (W3C), groups that have been working with hundreds of tech companies over the past decade to develop a new login standard that works the same way across multiple browsers and operating systems.

According to the FIDO Alliance, users will be able to sign in to websites through the same action that they take multiple times each day to unlock their devices — including a device PIN, or a biometric such as a fingerprint or face scan.

“This new approach protects against phishing and sign-in will be radically more secure when compared to passwords and legacy multi-factor technologies such as one-time passcodes sent over SMS,” the alliance wrote on May 5.

Sampath Srinivas, director of security authentication at Google and president of the FIDO Alliance, said that under the new system your phone will store a FIDO credential called a “passkey” which is used to unlock your online account.

“The passkey makes signing in far more secure, as it’s based on public key cryptography and is only shown to your online account when you unlock your phone,” Srinivas wrote. “To sign into a website on your computer, you’ll just need your phone nearby and you’ll simply be prompted to unlock it for access. Once you’ve done this, you won’t need your phone again and you can sign in by just unlocking your computer.”

As ZDNet notes, Apple, Google and Microsoft already support these passwordless standards (e.g. “Sign in with Google”), but users need to sign in at every website to use the passwordless functionality. Under this new system, users will be able to automatically access their passkey on many of their devices — without having to re-enroll every account — and use their mobile device to sign into an app or website on a nearby device.

Johannes Ullrich, dean of research for the SANS Technology Institute, called the announcement “by far the most promising effort to solve the authentication challenge.”

“The most important part of this standard is that it will not require users to buy a new device, but instead they may use devices they already own and know how to use as authenticators,” Ullrich said.

Steve Bellovin, a computer science professor at Columbia University and an early internet researcher and pioneer, called the passwordless effort a “huge advance” in authentication, but said it will take a very long time for many websites to catch up.

Bellovin and others say one potentially tricky scenario in this new passwordless authentication scheme is what happens when someone loses their mobile device, or their phone breaks and they can’t recall their iCloud password.

“I worry about people who can’t afford an extra device, or can’t easily replace a broken or stolen device,” Bellovin said. “I worry about forgotten password recovery for cloud accounts.”

Google says that even if you lose your phone, “your passkeys will securely sync to your new phone from cloud backup, allowing you to pick up right where your old device left off.”

Apple and Microsoft likewise have cloud backup solutions that customers using those platforms could use to recover from a lost mobile device. But Bellovin said much depends on how securely such cloud systems are administered.

“How easy is it to add another device’s public key to an account, without authorization?” Bellovin wondered. “I think their protocols make it impossible, but others disagree.”

Nicholas Weaver, a lecturer at the computer science department at University of California, Berkeley, said websites still have to have some recovery mechanism for the “you lost your phone and your password” scenario, which he described as “a really hard problem to do securely and already one of the biggest weaknesses in our current system.”

“If you forget the password and lose your phone and can recover it, now this is a huge target for attackers,” Weaver said in an email. “If you forget the password and lose your phone and CAN’T, well, now you’ve lost your authorization token that is used for logging in. It is going to have to be the latter. Apple has the infrastructure in place to support it (iCloud keychain), but it is unclear if Google does.”

Even so, he said, the overall FIDO approach has been a great tool for improving both security and usability.

“It is a really, really good step forward, and I’m delighted to see this,” Weaver said. “Taking advantage of the phone’s strong authentication of the phone owner (if you have a decent passcode) is quite nice. And at least for the iPhone you can make this robust even to phone compromise, as it is the secure enclave that would handle this and the secure enclave doesn’t trust the host operating system.”

The tech giants said the new passwordless capabilities will be enabled across Apple, Google and Microsoft platforms “over the course of the coming year.” But experts said it will likely take several more years for smaller web destinations to adopt the technology and ditch passwords altogether.

Recent research shows far too many people still reuse or recycle passwords (modifying the same password slightly), which presents an account takeover risk when those credentials eventually get exposed in a data breach. A report in March from cybersecurity firm SpyCloud found 64 percent of users reuse passwords for multiple accounts, and that 70 percent of credentials compromised in previous breaches are still in use.

A March 2022 white paper on the FIDO approach is available here (PDF). A FAQ on it is here.