The company’s latest Falcon Sensor software was meant to make CrowdStrike customers’ systems more secure from hacks by updating the threats it protects against, but a flaw in the update’s code caused one of the most widespread technology outages in recent years for companies using Microsoft’s Windows operating system.
Banks, airlines, hospitals and government agencies around the world were thrown into chaos. CrowdStrike has released information to repair affected systems, but experts said it will take time to bring them back online because the faulty code must be removed manually.
“Maybe because of the vetting or sandboxing that we do when we look at code, this file wasn’t included or slipped through,” said Steve Cobb, chief security officer at Security Scorecard, some of whose systems were affected by the issue.
Problems emerged quickly after the update was released on Friday, with users posting photos on social media of their computers showing an error message and a blue screen of death, known in the industry as the “Blue Screen of Death.”
Patrick Wardle, a security researcher specializing in threats to operating systems, said his analysis identified the code that caused the outage. He said the update problem was with “files that contain either configuration information or signatures.” Such signatures are code that detect certain types of malicious code or malware. “It’s very common for security products to update their signatures once a day. This is because they are constantly monitoring for new malware and making sure that customers are protected against the latest threats,” he said.
“The frequency of updates is probably why[CrowdStrike]didn’t test it as much,” he said.
It’s unclear how the flawed code got into the update, or why it wasn’t detected before it was released to customers.
“Ideally, they should have rolled it out to a limited number of people first,” said John Hammond, principal security researcher at Huntress Labs. “That’s a safer approach to avoid the chaos that we saw.”
Other security companies have seen similar incidents in the past: In 2010, McAfee released a buggy antivirus update that shut down hundreds of thousands of computers.
But the global impact of the outage reflects CrowdStrike’s dominance: More than half of the Fortune 500 companies and many government agencies, including the Cybersecurity and Infrastructure Security Agency, the top U.S. cybersecurity agency, use its software.
