Guest post by Cami Lewis, ArcSight Security Product Marketing Manager and Amir Einav, Head of Product Management, ArcSight 

Why is it that in 2017 it still takes large organizations weeks or even months to detect a major breach? SOC threat detection needs to move to the next level. Security Information and Event Management (SIEM) tools have used real-time correlation for more than a decade as the key approach to enterprise security operations, but SIEM tools are now evolving fast to leverage the latest technologies. In the past few years the proliferation of newer advanced analytics tools are now scouring infrastructures and alerting security personnel to more advanced threats that require investigation.  Here at ArcSight we believe that while there is tremendous value provided by SIEM, it is not enough.  Rule-based SIEM is a good foundational layer for many threats and especially prioritizing alerts but advanced threats might be missed even with a well-tuned SIEM. Exciting new advanced analytics and big data visualization provide new capabilities that together can tighten the detection grid and reduce the attack surface.

SOC teams around the world are facing concerns such a talent shortage in the millions as well as the exploding data variety and scale, but security operations practitioners are clearly aware that they have to improve their threat detection.  This is where the Detection Trinity comes in: a combination of real-time correlation, detection analytics and threat hunting.  These are three distinct types of detection techniques, all of which leverage enriched and normalized event log data, network traffic and end-point sensors.  Each technique by itself is capable of finding threats, while offering different considerations, but their combination is the detection Elysium of an advance Security Operations (SecOps).

Type of threats

Real-time correlation is very efficient in terms of compute resources and time-to-detect, but to reach that it requires expert developers and a powerful tool such as ArcSight ESM to be able to detect multi-stage attacks as well as low-and-slow attacks.  The expert use case developer understands what data sources are needed to identify potential threats and then program the logic into the tool to identify those outliers. The bread and butter of real-time correlation is to fast expose known threats in massive quantities so it doesn’t exhaust human resources.  Detection analytics, which apply algorithms in the areas of event anomaly and entity relationship on specific data sources, can expose the unknown threats within certain use cases domains. By definition, the alerts are statistical in nature and advance tools have a feedback loop to re-caliber the sensitivity of the anomaly engine.  Both real-time correlation and detection analytics are automatic detection tools that cover specific use cases, however applying different techniques to identify threats.  Developing the set of use cases that need to be covered, and their relationship to the set of data sources available for the SOC, can be based on various conceptual models (e.g. Defense in Depth, Attack Life Cycle, etc.) as well as by leveraging vendors “use cases catalogue” and sometimes business units’ unique requests.

Hunting, on the other hand, is a very different detection technique.  Hunt is focused on free-form investigation in which the hunter is following threads and hypothesis to detect unknown threats. Many times those hypothesis start from a threat Intel activity or sources and thus focus on the forefront of advance threats. Investigating a hypothesis with full access to massive amounts of data and leveraging powerful visualization tools are allowing the hunter to expose attack vectors not yet conceived by the automatic detection tools.  

Time to Detect

A key difference between the three detection techniques is speed.  While real-time correlation is “shallower” in terms of detection depth, it is extremely fast and accurate as the analyst is investigating an event in real-time.  Further, solutions like ArcSight ESM are providing the analyst with ability to set “monitors” which are updated in real-time as more correlation related to the investigation topics are revealed, seeing the attack progress.

Detection analytics, done in the traditional batch mode, is somewhat slower as the size of the batch and the depth of the computation is taking its toll. However, by strongly integrating with real-time systems it can leverage the algorithms insights to provide high fidelity results. 

Hunt, by nature, is a slow manual process, either using methodologies and best practice or providing the hunter an open field to explore the data and “massage” it. However, hunt compensates for this shortcoming with depth. Strong tools, such as ArcSight Investigate, allowing for application of analytics algorithms and smart visualizations within the hunt process to speed up the potential insights.

Costs

Finally, when looking at the cost associated with the three detection techniques, one should assess the initial investment as well as the on-going operational costs.  Fast and powerful correlation requires a certain type of hardware and network infrastructure while big data analytics solutions, that can be used for detection analytics and hunting, will focus on a large amount of storage nodes (“big data”). Modern architecture, in any case, will strive for a scalable model that can expand, on-demand, the hardware needs (storage, compute) for each detection technique.

There is also the consideration of quantity and cost profile of employees needed to operate each model.  How do you develop the talent to support the detection tools? Growing a separate team of hunt professionals, for example, or correlation rule developers can take time and effort. Some of our customers have leveraged tools like ArcSight Investigate to develop a smooth transition between SOC analysts and hunters, keeping the learning curve costs reasonable.

Conclusion

A mature SOC is looking to shorten time-to-detect by applying multiple techniques. Considerations as to which technique to introduce and in what order include budget constraints, vendor relationship and potential talent at hand.  Amount of data collected and how to manage it, by security or together with IT, is also a major concern. A well-planned solution will leverage the same data collection infrastructure, which then creates a single reference point for all threats identified, as well as focusing on methods to integrate one detection technique with another, thus, building a learning organization.