Cloudflare announces that there was unauthorized access to the server hosting internal wiki etc., analysis and response has been completed, there is no impact on user data or system
Cloudflare announced on November 23, 2023 that there had been unauthorized access to a server that hosts internal wikis, and at the same time posted the analysis results on its blog.
Thanksgiving 2023 security incident
The attack was on Cloudflare's self-hosted Atlassian server, which hosts internal wikis and other resources. After detecting the attack on November 23, 2023, Cloudflare's security team immediately blocked the attacker's access and began an investigation. On November 26, three days after the attack, CrowdStrike 's forensic team was brought in to conduct an analysis.
The investigation was completed on January 31, 2024. Our investigation reveals that access controls, firewall rules, and hardware security keys limit the ability to move laterally across the company's network, and that no Cloudflare user data or systems were affected by the attack. It is said that it has become.
The timeline of the attacks is as follows.
◆October 18, 2023
The systems of
・Moveworks service token for remote access to Atlassian systems
- A Smartsheet service account with administrative access to your Atlassian Jira instance
・Bitbucket service account for access to source code management system
・Credential information for AWS that does not have access rights to the global network and does not contain users or confidential data.
Cloudflare stated that these credentials were not changed because they were ``erroneously recognized as unused.''
◆November 14, 2023 9:22:49
Using the above credentials, the attackers began probing and reconnaissance on Cloudflare's systems, looking for ways to use the credentials and what systems they could access. There are logs of login failures to Okta instances and access to the Cloudflare dashboard being denied.
◆November 15, 2023 16:28:38
The attacker used a Moveworks service token to authenticate through the gateway and was able to access Atlassian Confluence, which Cloudflare uses as an internal wiki, and Atlassian Jira, a bug database. He also said that he began accessing the entire Atlassian system using the Smartsheet service account.
The attackers searched for information about configuring and managing Cloudflare's global network, accessing 36 of 2,059,357 Jira tickets and 202 of 194,100 internal wiki pages. The content accessed was primarily Cloudflare's Jira tickets related to Cloudflare's response to the Okta credential leak in October, as well as vulnerability management, secret rotation, MFA bypass, network access, password resets, and remote access. It is said that the target was system-related and did not target user data or user configuration.
◆November 16, 2023 14:36:37
The attacker used Smartsheet credentials to create an Atlassian account that appeared to be a regular Cloudflare user. The attackers then added this account to several groups within Atlassian, allowing it to access the Atlassian environment even if the Smartsheet service account is removed.
◆November 22, 2023 14:18:22
Attackers used ScriptRunner for Jira to install a framework that allowed them persistent stealth access to Atlassian servers. Afterwards, the attacker attempted to access the Sao Paulo data center, which had not yet started operations and had no access controls in place, but was unsuccessful.
The next day, the 23rd, 120 out of 11,904 code repositories were viewed. These repositories are related to backup mechanisms, global network configuration and management, ID mechanisms, remote access, and the use of Terraform and Kubernetes, and some had secrets written in the code.
◆November 23, 2023 15:58~
At 15:58, the attacker added the Smartsheet service account to the Administrators group, which triggered an automated alert to the security team at 16:00. Cloudflare's Security Operations Center (SOC) began an investigation at 16:12, and the Smartsheet service account was deactivated at 16:35. At 17:23, an Atlassian user account created by the attacker was discovered and has also been deactivated.
An internal Cloudflare incident was declared at 17:43, and a firewall rule was set to block the attacker's known IP address at 21:31. The next day, at 11:59 on November 24th, the stealth access framework installed on the Atlassian server was removed, and the attacker's activities ceased.
The attackers attempted to gain access to various other Cloudflare systems during the series of attacks, but they did not have access controls or firewall rules in place, or proprietary Zero Trust tools that enforced the use of hardware security keys. All accesses failed due to some errors. 'The attacker's access was limited to Atlassian products and the servers Atlassian was running on,' Cloudflare concluded.
Although this incident has had little to no impact on service operations, Cloudflare takes the fact that internal documents and source code were accessed seriously and is working hard to limit the impact of the incident and take sufficient steps to prevent future advanced attacks. He said that he had been working on ensuring that preparations were being made as a top priority task for more than a month.
Testing revealed no problems with the equipment at the Brazilian data center that the attackers were attempting to gain unauthorized access to, but to ensure that it is 100% secure, the hardware must be returned to the manufacturer and replaced with new hardware. Apparently it was exchanged for something else.
As for the identity of this attacker, Cloudflare says, ``It is likely a state-run effort to gain permanent and widespread access to Cloudflare's global network.''
Related Posts:
in Web Service, Security, Posted by log1d_ts