On April 20^th, 2021 MITRE Engenuity released the results from the 2020 ATT&CK® Evaluations, in which ArcSight ESM was a first-time participant. Historically, this has been the domain of the Endpoint Detection and Response (EDR) products, but ArcSight’s whole-hearted embrace of the MITRE ATT&CK® framework meant we weren’t going to shy away from the challenge. We are proud to be one of the first SecOps vendors to participate and show another point of view; how a pure-SIEM play could enhance EDR results, by providing additional visibility. This also shows the meticulous work we’ve put in to align ArcSight with MITRE over the past years, and we’re pleased with the results!

Why participate in the ATT&CK® Evaluations?

In the 19th century, the Rosetta Stone helped scholars at long last crack the code of hieroglyphics, the ancient Egyptian writing system. ATT&CK is the Rosetta Stone for modern defenses, helping the defense industry crack the code of the sophisticated attacks.

ATT&CK Evaluations, takes it one notch further, by acting as an objective and unbiased reference tool to understand the current state of the vendors.

“At ArcSight, we firmly believe in independent third-party testing and verification of defenses. This is what the cybersecurity industry has been lacking for so long. Exposing the products to MITRE’s sophisticated red teams is the only *known* way to get tested against today’s APT’s that we observe in the real-world every day. Micro Focus ArcSight proudly partners with MITRE on this journey to make the world a safer place.”  - Emrah ALPA, Product Manager, Micro Focus ArcSight.

How did ArcSight ESM do?

The 2020 evaluation used the MITRE ATT&CK® knowledge base to emulate the tactics and techniques of Carbanak and FIN7, both of which have compromised financial services and hospitality organizations through the use of sophisticated malware and techniques, resulting in the theft of more than $1B across hundreds of businesses over the past five years. 

Despite the complexities involved in entering our SIEM solution in a primarily EDR evaluation, especially when tested against Carbanak and FIN7 tactics and techniques, we’re happy with our performance. “Considering we are a SIEM that does not use or require an endpoint presence and the program is primarily designed to test endpoints, I believe this proves that SIEMs—when properly configured—can provide realistic and tangible value in this area.  We demonstrated that the past two years of work paid off in a meaningful way and that customers consuming our content should feel confident we can provide a strong layer of defense for these types of attacks.” - Michael Mychalczuk, Director of Product Management, Micro Focus Security Operations.

Was ArcSight Intelligence part of this evaluation?

It’s important to note that ArcSight Intelligence was not used for anomaly detection in the 2020 ATT&CK® Evaluations. “ArcSight Intelligence is a powerful supplement to ArcSight ESM, giving organizations even more comprehensive and holistic coverage of the MITRE ATT&CK® matrix. By using anomaly detection and unsupervised machine learning, ArcSight Intelligence automatically learns normal behaviors in your environment, for every user, machine and device, so even the most advanced or subtle MITRE techniques and tactics can be detected. The Carbanak and FIN7 tests only had three days of red team activity, so there is no 'normal data' for ArcSight Intelligence’s AI technology to adequately learn behaviors, so we opted not to deploy ArcSight Intelligence in this artificial environment. Of course, this is not a problem in real-world environments, for the detection of real-world threats; we've found the combination of ArcSight ESM and ArcSight Intelligence to be an unbeatable partnership for the SOC!” Stephan Jou, Chief Technology Officer of Interset, Micro Focus.

What has ArcSight gained as a result?

When asked this question, Emrah Alpa, Product Manager of ArcSight Content at Micro Focus responded: “By taking the road less travelled, we showed the SIEM industry which original and unique ways of collecting endpoint, and non-endpoint (like network, DNS…) exist, in order to detect today’s advanced threats.

Update on 2021.05.10:
"We'd like to make sure there's no misunderstanding on our use of the word "endpoint logs". We did not use any endpoint detection engine, or a third party EDR solution to achieve our results. Even the Windows Defender that comes enabled out-of-the-box with Windows 10, had to be turned off, as a rule of the ATT&CK Evaluations engagement."

As a result, our team used the default endpoint logs that were present in any Windows and Linux hosts, called "live off the land" tools, that is also a term used in the attack landscape.
These included such log sources as PowerShell, Sysmon, WMI and Process Creation Auditing events, to name a few.
For full details of the log source selection and configuration, please see the Marketplace post at the end of this blog post.

It was not just sunshine and unicorns, though, as the goal of detecting “only” the 66 x ATT&CK techniques commonly used by financial sector’s most notorious threat actors Carbanak and FIN7, was only the tip of the iceberg. 4 months, 2 teams, 250+ correlation rules and countless sleepless nights later, we understood exactly what it meant to form a red, blue and a purple team to collectively improve our own tradecraft.

Through this team approach, we were able to advance the ArcSight product’s detection capabilities, focusing on the TTPs of the attackers all the time, in order to provide a long-term-lasting ecosystem of baseline correlation rules.

Arming the product with strong and hard-to-evade correlation rules, automatically enables ArcSight-based SOC’s to achieve higher visibility in today’s advanced attack landscape.

Not interested in Carbanak or FIN7?

No problem. Our customers can still use this solution package as a starting point to build their defenses. Next step would be to customize them to match any other threat actor that were of interest.

In fact, we have built Micro Focus’s ArcSight ATT&CK Landing Page exactly for this purpose, where you can export the detections as a JSON file, then import it into MITRE ATT&CK Navigator as a 2^nd tab, and then compare a custom selection of techniques to that of ArcSight. This would allow our customers to quickly see what threat actors they could identify on Day 1, using the Default Content that came out-of-the-box with ArcSight. Through iterative deployment, higher maturity levels would be achieved in time.”

Conclusion:

Overall, we felt this was a great opportunity for ArcSight to boldly step up to the challenge in order to test and refine ArcSight ESM’s detection and correlation capabilities. As one of the few SIEM vendors in this year’s MITRE ATT&CK® Evaluation, we are happy to be on the forefront of SIEM adoption of the ATT&CK® framework.  ArcSight has already developed downloadable content as a direct result of this experience, and we’re grateful to Engenuity for making the world a safer place!

Additional Resources:

• ArcSight’s Results
• ArcSight’s Alignment with MITRE ATT&CK® Framework
• ArcSight Content for Carbanak and FIN7

Have technical questions about Security Operations? Visit the ArcSight User Discussion Forum. Keep up with the latest Tips & Info about Security Operations. Do you have an Idea or Product Enhancement Request about ArcSight? Submit it in the Idea Exchange. We’d love to hear your thoughts on this blog. Log in or register to comment below.