MITRE Engenuity ATT&CK® Evaluations (Evals), a program of MITRE Engenuity™, MITRE’s tech foundation for the public good, announced the results of its first-ever independent ATT&CK Evaluations for security service providers. The evaluations highlighted results across 16 providers and assessed provider capabilities in their ability to analyze and describe adversary behavior.
“More than half of organizations use security service providers to protect their data and networks. We wanted to research how they are employing threat-informed defense practices for their clients,” said Ashwin Radhakrishnan, general manager, of ATT&CK Evaluations, MITRE Engenuity. “We don’t rank the vendors in our evaluations. Organizations, however, can use the evaluations to determine which service providers may best address their own cybersecurity gaps and fit their particular business needs.”
Evals’ expert purple teamers have in-depth knowledge of the threat landscape and adversary tradecraft. Through the lens of the MITRE ATT&CK knowledge base, the team emulated the tactics and techniques of OilRig, a threat actor with operations aligning with the strategic objectives of the Iranian government. OilRig has conducted operations relying on social engineering, stolen credentials, and supply chain attacks, resulting in the theft of sensitive data from critical infrastructure, financial services, government, military, and telecommunications. This threat actor used in evaluating the security service providers was chosen based on its evasion and persistence techniques, its complexity, and its relevancy to the industry.
Participants in the evaluations included Atos, Bitdefender, BlackBerry, BlueVoyant, Critical Start, CrowdStrike, Microsoft, NVISO, OpenText, Palo Alto Networks, Rapid7, Red Canary, SentinelOne, Sophos, Trend Micro, and WithSecure.
Background on Tracking Confidence in Security Service Providers
Prior to the evaluations in 2021, MITRE Engenuity conducted research with Cybersecurity Insiders, an online community of more than 400,000 information security professionals worldwide, to understand the state of affairs in security services. The 2021 Managed Services Report, No Rest for the Wary, found that most respondents (68%) used security services, yet nearly half (47%) were not confident in the service technology or people. At the same time, when asked whether teams conduct offensive testing before the selection process, 59% of respondents claimed to conduct offensive testing on products while only 53% conducted testing on services.