Following the major CrowdStrike outage on July 19 and the flaws discovered in Google Chrome’s update on July 24, there is an increasing need for more rigorous checks on software update vulnerabilities.
The CrowdStrike outage, according to the company’s website, occurred because an automated software failed to detect an error in a problematic update. The company uses a tool called Content Validator to ensure system functionality. However, a bug in the Content Validator led to the validation of software containing faulty content.
The severity of the CrowdStrike outage was significant because CrowdStrike, a business partner of Microsoft, had direct access to the Windows operating system. This access was granted to enable the cybersecurity software provider to manage clients promptly. CrowdStrike utilized this access to implement their Rapid Response Content (RRC) for real-time detection of system malfunctions. However, this same access contributed to the widespread impact of the faulty update.
Google Chrome also experienced issues due to a problematic update. On July 24, users were unable to find or add passwords in their Password Manager for nearly 18 hours. According to the incident report, the problem stemmed from “a change in product behavior without proper feature guard.” Google did not disclose the specific cause of the inadequate examination of the update.
In an email interview, Kim Eui-tak, Vice President of Ui Networks, noted that detecting software vulnerabilities has become increasingly challenging. Modern software often integrates with numerous other systems and applications, raising the risk of potential vulnerabilities. Additionally, the growing volume of code makes errors more difficult to identify. Kim emphasized that even with rigorous Quality Assurance (QA) and Quality Control (QC) processes, achieving flawless software remains a significant challenge.
Kim also proposed solutions to prevent update flaws. He pointed out that if companies focus solely on testing software performance, there is a greater possibility that they will overlook security risks. He stressed the need for more comprehensive assessments to address vulnerabilities such as these. Kim warned that inadequate software testing could lead to business disruptions, and failures in antivirus updates could harm businesses, public services, and financial institutions. Additionally, he highlighted the importance of prompt responses to emerging issues.
To prevent defective updates, CrowdStrike has outlined several measures. They plan to enhance their Rapid Response Content (RRC) by adopting a staggered deployment strategy, which involves rolling out updates in smaller increments rather than all at once. To improve software resilience, CrowdStrike will implement local developer testing, rollback testing, stress testing, fuzzing, and fault injection.
Local developer testing involves developers evaluating the software in a controlled environment. Rollback testing allows the software to revert to a previous, error-free version if needed. Stress testing, fuzzing, and fault injection involve testing the software under extreme conditions, including handling inputs beyond the software’s limits, processing erroneous data, and forcing the software to operate in challenging environments.
Another crucial aspect of validating software is third-party validation. Even with thorough internal checks, some errors may still be overlooked. To address this, CrowdStrike incorporates multiple third-party reviews to assess code security and evaluate end-to-end processes, from development through to deployment.
In an interview on August 5, Professor Ko In-young, a software engineering specialist at the Korea Advanced Institute of Science and Technology (KAIST), discussed solutions to mitigate update risks. Ko highlighted the challenges of detecting issues due to the complexity of integrated systems. He emphasized that rollback-testing and third-party validation are critical for ensuring software reliability. Additionally, he noted the importance of gradually implementing updated versions and stressed that thorough discussions among stakeholders are essential, particularly alongside rigorous software testing.
Related article: Financial Security Institute emphasizes security strategies following CrowdStrike global outage
The Financial Security Institute (FSI) of South Korea has raised concerns following the global outage incident involving CrowdStrike. The FSI emphasized that South Korean financial firms need to be better prepared for digital incidents to ensure the stability of the financial system.
On Monday, the FSI held a seminar for financial companies to review their security measures. The event, which took place on Wednesday, included five security experts from banking, securities firms, and academia. They convened to discuss the necessary countermeasures for South Korea’s financial sector in response to a recent global outage incident that affected approximately 8.5 million IT devices worldwide.
Kim Chul-woong, President of the FSI, emphasized that the CrowdStrike incident highlighted the vulnerability of the entire industrial ecosystem. He underscored the importance of security preparedness, noting that, although the security damage to Korea was minimal, the potential impact on the broader system is significant. READ MORE
