Planes Grounded, Businesses Disrupted Due to Tech Issue

Probably an ignorant question here, but.....if we still had Windows boot disks, could we not have booted up off of the disks into a Safe Mode and gone in and deleted the *.SYS file or at least the updated version? Seems like the over-automation and insisting everything be on the cloud by Micro-soft exacerbated the problem??

 

I know it wasn't directed at me, but that's kinda the issue in a nutshell. For an enterprise size organization with 10s of thousands of devices, that it an absolute ton of hands on time booting into a recovery mode to fix something, something you're not doing remotely or using an automated job to do.

But yes, what you mentioned would be a fix.

 

Papa Johns came through, no worries

 

I laughed when I saw null pointer exception, my team has done the same thing to an important nationally used system in the past, it’s easy to do. Thankfully ours only triggered on rare boundary scenarios (which was why we didn’t catch it), and a back out was easy. What boggles me here is if it was that prevalent to literally bring down half the planet, how it wasn’t caught in pre-release testing. There must be an explanation.
Meantime, I am sure their team retro on this will be fun, with no defensiveness, finger pointing or anger at all .

 

Couldn't resist, just bought in for my first position. Hopefully it finds a bottom soon. If it drops another 5%, I'll buy some more. Reaction seems extremely overblown

 

And I just read an article that Delta is on day 4 of heavy cancellations/delays, because their crew scheduling system is still not fully functional. I still see significant reputational damage for Crowdstrike here... many of the impacted companies are probably already looking into alternatives. How many pull the trigger remains to be seen.

 

Southwest is thinking what do you think about my windows 3.1 system now?? lol

 

Mr Swampbabe’s flight got cancelled this morning out of MCO, no pilot. Said there were THOUSANDS of pieces of luggage stacked up and a whole bunch of angry people yelling at the Delta gate agents. Couldn’t get him out until Wednesday, he’s on his way home now. What a mess.

 

I am supposed to fly on Delta tomorrow.

 

Bunch of my neighbors work for Delta, they are definitely struggling. The crew management portion of crowd strike definitely screwed them over.
I made it back from Munich the day before this all went down, so I was very lucky. Flying out next week to Jackson Hole, I believe things will be smoothed out by then.

 

Yeah they blamed their testing process too.
Pretty mind boggling this is what it took for them to put decent testing guardrails in place.

CrowdStrike pledges to improve testing after IT outage

 

Crazy mismanagement over there....

In my company, we have a concept known as "controlled rollout". Certain trusted customers get first access to a software release to help validate before we go wide with it. Of course, that's after we do in-house testing (scripted and non-scripted), automated testing, alpha tests, beta tests. You name it. And we still wind up with egg on our face every once in a while when some corner case goes off the rails.

 

When the news broke my first thought was a hack by Russia, NK, China et al. Seems it doesn't take much to gum up the works in a major way. Yet another reason flying sucks, though sometimes it is the only practical way to get somewhere.

 

I thought I was living dangerously by still using XP on a couple machines around the house. Windows 3.1 stopped being supported in 2001.

But I totally get the idea of if it ain't broke, don't update it.