Can you imagine that a reputable organisation would deploy a business-critical security service without first designing and testing it, and then reviewing it to ensure that it operated as expected? Or, would you expect an organisation to allow a security service that was not well-designed, tested and regularly reviewed to become the cornerstone of the organisation’s critical security infrastructure?
And as if those questions were not rhetorical enough, can there be any doubt about the organisation’s fate if the critical security service in question is a firewall or remote-access server?
Of course not!
So, if we accept that critical security infrastructure like firewalls and remote-access servers should be well-designed, tested before deployment and then reviewed on a regular basis, it should be implausible that similarly-critical Security Information and Event Management (SIEM), Security Operation Centre (SOC) and Incident Detection and Response (IDR) infrastructure would be treated differently!
Implausible you say? Read on!
Let’s have a look at just one example of how bad things can get if the SIEM/SOC/IDR doesn’t work as intended. Back in 2019, the Marriott International hotel chain discovered that attackers had breached one of its reservation systems; that system (which originally belonged to Starwood, a hotel chain that was acquired by Marriott in 2016) was breached in 2014, and remained breached for another four years!
As we’ve noted elsewhere, IBM’s reporting shows:
Of course, that 220-day average is insignificant when we see that the Starwood/Marriott data breach remained unidentified for over four years!
So, back to our first rhetorical questions: Would you expect an organisation to allow a security service that was not well-designed, tested and regularly reviewed to become the cornerstone of the organisation’s critical security infrastructure? Well in the case of the Starwood/Marriott breach, the plaintiffs in a multidistrict litigation (MDL) are saying that the answer is ‘no’ and their case includes specific claims regarding the effectiveness of the SIEM/SOC/IDR:
No one can say what damages may be awarded should the MDL succeed but we can hark back to the 2017 Equifax breach which affected about 147 million people. Equifax’s ability to detect attackers was also raised as an issue with the Federal Trade Commission’s complaint for permanent injunction and other relief alleging: “the Defendant had minimal protections for detecting intrusions on ‘legacy’ technology systems […] which contributed to Defendant’s months-long failure to detect the attackers on its Network”.
The eventual settlement included the creation of a US$380 million fund by Equifax, payment by Equifax of US$77 million in attorney’s fees, and provision by Equifax of years worth of credit monitoring and identity protection and restoration services.
Call us crazy but at DotSec, we’d prefer to avoid the kind of expense and publicity that (as outlined above) accompany the failure of a SIEM/SOC/IDR service. To achieve that goal, we initiate regular Purple Teaming (also known as Collaborative Red Teaming) exercises on both our Managed Security Information and Event Management (MSIEM) infrastructure, and against the infrastructure that is owned by our MSIEM clients.
The idea behind purple teaming is to assume a breach has already taken place. After all, a pentesting team, being limited by time/budget may not be able to identify all places where an initial compromise may take place.
Starting from this position, we build two separate groups; the Red (or attack) team and the Blue (or defence) team. The two teams cooperate (that’s very important!) to run a campaign to exercise standard Tactics, Techniques and Procedures (TTPs) in the defender’s environment with the aim of identifying which TTPs get blocked/detected by the Blue team. In concrete steps:
That’s good! Penetration testing (pen testing) can certainly help an organisation to understand how its vulnerabilities can be detected and exploited, and this is a good thing. However for organisations that have been pen tested to death, purple teaming can be very illuminating! For example, by starting from an ‘assume breach’ position and using TTPs which are known to both attackers and defenders, you can test your SIEM/IDR capabilities through the entire lifecycle of a breach.
Compare this to a penetration test where the main goal is to uncover vulnerabilities leading to an initial compromise, with perhaps further penetration into the target’s environment. This would be a useful exercise but it may not exercise the target organisation’s SIEM/IDR capabilities at all. And even if it does, the scenarios that are exercised will only be those that correspond to the vulnerabilities that are found and exploited by the pen tester.
By contrast, a purple teaming exercise assumes that a breach has taken place and is therefore focussed on exercising the SIEM/SOC/IDR capabilities of the target organisation. And there are so many fun ways to do this! DotSec currently has over 30 adversarial campaigns available for use, and is constantly creating new campaigns that are based on notable attacks that are making headlines.
As an example, one of our purple teaming exercises emulates an adversary that uses the following TTPs:
While the Marriott MDL continues through the US court system, the UK Information Commissioner’s Office issued a fine of 18.4 million pounds (about A$33M) against Marriott (under the GDPR) for the Starwood/Marriott breach. As the ICO described, Marriott was penalised for:
Performing regular validation of their SIEM/SOC/IDR capabilities would have definitely helped Marriott to detect and remediate at least the first three of the above four (75%!) deficiencies.
As mentioned at the beginning of this article, no one would (reasonably) expect an organisation to allow a security service that was not well-designed and/or tested and regularly reviewed to become the cornerstone of that organisation’s critical security infrastructure.
SIEM/SOC/IDR is critical security infrastructure and it needs to be regularly validated against changing IT environments to make sure it’s still going to be effective when you really need it. Running regular Purple Teaming exercises is one of the simplest, most cost-effective (and low-publicity!) ways to achieve that goal.
PS: It’s happened again! This time with Neiman Marcus.