What went wrong in CrowdStrike's code that caused so many Windows users to experience blue screens?

CrowdStrike engineer Patrick Wardle analyzed the cause of the problem that caused blue screens of death on Windows PCs around the world and posted a post on X.

I don't do Windows but here are some (initial) details about why the CrowdStrike's CSAgent.sys crashed

Faulting inst: mov r9d, [r8]
R8: unmapped address

...taken from an array of pointers (held in RAX), index RDX (0x14 * 0x8) holds the invalid memory address

@_JohnHammond pic.twitter.com/oqlAVwSlJj

— Patrick Wardle (@patrickwardle) July 19, 2024

The CrowdStrike-related Blue Screen of Death issue affected 8.5 million Windows devices and caused several public transport systems to stop functioning.

CrowdStrike's Blue Screen of Death issue affected 8.5 million Windows devices, less than 1% of the total - GIGAZINE

According to Wardle's analysis, the cause of the vulnerability was an invalid memory address specified in the 'mov r9d, [r8]' command in the CrowdStrike executable file 'CSAgent.sys.'

As mentioned above, the application that specified the invalid address and directly caused the crash was 'CSAgent.sys'. However, the data for the invalid address was stored in a file called 'C-00000291-[name varies depending on the environment].sys'. CSAgent.sys crashed because it read data from the C-00000291~.sys file.

The other 'drivers' (eg 'C-00000291-...32.sys') appear to be obfuscated data ...and are x-ref'd (perhaps ingested?) by CSAgent.sys

...so maybe invalid (config/signature) data triggered the fault in CSAgent.sys

This would be easier to tell/confirm via debugging 😅 pic.twitter.com/AEOAFVT40i

— Patrick Wardle (@patrickwardle) July 19, 2024

Therefore, the Crowdstrike problem can be resolved by booting into safe mode and deleting C-00000291~.sys.

How to fix the Crowdstrike thing:

1. Boot Windows into safe mode
2. Go to C:\Windows\System32\drivers\CrowdStrike
3. Delete C-00000291*.sys
4. Repeat for every host in your enterprise network including remote workers
5. If you're using BitLocker jump off a bridge

— vx-underground (@vxunderground) July 19, 2024

According to CrowdStrike's official blog , C-00000291~.sys is a type of configuration file called a 'channel file,' which is updated several times daily in response to new tactics, techniques, and procedures discovered by CrowdStrike.

An update from @CrowdStrike confirms our analysis: https://t.co/2BpWVqOuMm

Namely:
▫ The C-...sys files aren't kernel drivers, but rather are 'configuration files' dubbed 'Channel Files'

▫ C-00000291- 'triggered a logic error that resulted in an OS crash' (via CSAgent.sys)

— Patrick Wardle (@patrickwardle) July 20, 2024

C-00000291~.sys is an automatic update file, and the scale of this accident was magnified because it was automatically applied to all clients regardless of their settings.

Note 'channel updates ...bypassed client's staging controls and was rolled out to everyone regardless' https://t.co/UecaAmJdqc

A few IT folks who had set the CS policy to ignore latest version confirmed this was, ya, bypassed, as this was 'content' update (vs. a version update)

— Patrick Wardle (@patrickwardle) July 19, 2024

CrowdStrike's official blog states that it is conducting a 'thorough root cause analysis to determine how the logic flaw occurred' and that it will publish the results of the root cause analysis as the investigation progresses.

It has also been revealed that CrowdStrike had caused an issue in April 2024 that rendered PCs unable to boot on Debian and Rocky Linux.

CrowdStrike's PC boot failure problem has previously occurred on Debian and Rocky Linux - GIGAZINE

◆ Forum is currently open
A forum related to this article has been set up on the official GIGAZINE Discord server . Anyone can post freely, so please feel free to comment! If you do not have a Discord account, please refer to the account creation procedure article to create an account!

• Discord | 'CrowdStrike caused a global PC outage. Did it have any impact on you?' | GIGAZINE
https://discord.com/channels/1037961069903216680/1264874986376659086