Crowdstrike security software had an update issue that took down many businesses globally. Supposedly fixed and cascading to businesses. I have coworkers stuck at airports and most of my company's systems didnt update last night. Going to be a pain if you are traveling. Saw a report 1/3 of McDonald's in Japan were closed due to register issues. Is this a test run for Skynet? Live updates: Microsoft global outage hits airlines, banks and businesses | CNN Business Global outages: Tech disruptions worldwide have hit airlines, banks and businesses, which are scrambling to respond. What’s behind this? The outages appear to stem at least partly from a software update for Microsoft Windows operating systems issued by cybersecurity firm CrowdStrike, experts tell CNN. Fix deployed, but impacts remain: CrowdStrike’s CEO said that a fix has been deployed. And Microsoft said the “underlying cause” has been fixed, but that the outages are still affecting some services. What has been affected: Major US carriers, including Delta, United and American Airlines, have had flights grounded, and airlines in Europe and Asia-Pacific region have also seen disruptions. Banks in Australia, New Zealand, South Africa and Britain have been impacted, as have health services in Israel and the UK.
Alaska's 911 service is/was down. ESPN was unable to air live programs (SportsCenter was not aired this AM.) Airports worldwide could not scan passengers. Reports of surgeries being cancelled and patient records not loading. My phone was blowing up before I got to the office asking for us to check all systems. Oddly, my office laptop had updated overnight and had to be restarted which made me nervous. What a mess.
Have not seen an RCA yet, but Crowdstrike already put out a fix and rebooting affected windows instances ‘should’ fix the issue.
Affected 911 here in volusia. Computers down. Also my bank log in is fine but investment firm website is down
Simple instructions as to how to fix from someone I am friends with that their company sent out . She is a non-tech HR person. I mean, I cant even. 1. Cycle through your blue screens until you get the recovery screen. 2. Navigate to "Troubleshoot>Advanced Options>Startup Settings 3. Press "Restart" 4. Skip the first Bitlocker recovery key prompt by pressing Esc 5. Skip the second Bitlocker recovery key prompt by selecting Skip This Drive in the bottom right 6. Navigate to Troubleshoot>Advanced Options>Command Prompt 7. Type bcdedit /set {default} safeboot minimal 8. Hit enter 9. Go back to the WinRE main menu and select Continue 9. It may cycle 2 to 3 times 10. If you booted in safe mode, log in as normal 11. Open Windows Explorer, navigate to C:\Windows\Systen32\drivers\Crowdstrike 12. Delete the offending file (STARTS with C-00000*.sys file extension) 13. Open command prompt (as administrator) 14. Type bcsedit /deletevalue {default} safeboot 15. Press enter 16. Restart as normal, confirm normal behavior
IT guy here... They done goofed. It sounds like this hasn't been automated yet and every device will need to be touched. Yikes.
Thanks to ValdostaGatorFan for the IT perspective. From a Risk perspective this is one reason to have Cyber insurance as it can be written (but is not always) to cover "dependent business interruption" - in essence, paying for your loss of income due to a connected party of yours having an issue. Stay safe out there and don't fall for the urgent need from "your CFO" demanding a wire be sent while he/she is about to go on vacation. Thieves are clever.
A part of a risk assessment, or insurance in general, should include regular in-house and 3rd part pentesting. A lax security posture should lead to a company being un-insurable. Sadly, in this case, companies were using Crowdstrike, which is good, but Crowdstrike really mucked up. Similar to the SolarWinds Orion supply chain attack, sometimes even doing the right thing ends poorly.
or that is the story and an official actor f'd us up on purpose. hide in the crowdstrike update and then deploy once rooted in? crowdstrike has made lots of enemies and I assume their talent pool isn't squeeky clean...???
That was the mechanism of the SolarWinds hack. Supply chain attack. I have a hard time believing that it wasn't thoroughly tested before deployed, but ish happens. IMO, if a nefarious actor inserted code into the update before it was deployed, they'd do a lot more damage than some bluescreens. Time will tell, though. I just got a call from my coworker, also IT, and he's stuck in the ATL airport. He called me after seeing several BSODs and recovery screens. Yikes. Another co-worker, non IT, is stuck in Wisconsin because of issues at an airport there.
