Celebrating 20 Years of ArcSight and the Evolution of SIEM

by in Security

Celebrating 20 Years of ArcSight and the Evolution of SIEM.png

This year marks the 20th birthday of ArcSight! ArcSight was initially founded in Delaware on May 3, 2000, as Wahoo Technologies, Inc before officially changing its name to ArcSight, Inc. in March the next year. Around 2002, ArcSight released its first product, and was later listed as a visionary in Gartner's "IT Security Management Magic Quadrant" 2003 report, where there were no leaders at that time. ArcSight’s initial public offering (IPO) went live on Feb. 14, 2008, trading under the NASDAQ symbol ARST.


A couple of years later, Hewlett-Packard (HP), completed its acquisition of ArcSight on Oct. 22, 2010, and by 2015 HP separated to create two companies – Hewlett Packard Enterprise (HPE), and HP Inc. In this split, ArcSight moved to be part of the newly created HPE. A year later, in 2016, HPE announced that it would be merging its software business with Micro Focus International (MFI) finalized on Sept. 1, 2017, in which ArcSight was included.

I was initially introduced to ArcSight in October 2005, when I started a new job as a security analyst. The ArcSight ESM version was ESM 2.5, yet it was upgraded shortly after to version 3.5. Before that job, I worked in various roles such as Intrusion Detection Analyst, Systems Administrator, or Network Administrator, just to mention a few. I still remember BNC connectors, polishing fiber optics ends, the first time I saw a CISCO Pix, Novell 3.x, Windows 3.x, or NT4.0. Also, I remember creating custom scripts while performing incident investigations.

Later, I moved from the analyst chair to maintain the Security information and event management (SIEM), as well as intrusion detection system (IDS) solutions, and I needed to create new content as new indicators emerged continually. After, I moved into the implementation and architecture side, and I finally ended up joining ArcSight in June 2010. By that time, I already had various successful SIEM deployments, including ArcSight. I continued following ArcSight through HP, HPE, and MFI moves with so many implementations, and customers throughout those years that they are hard to keep them straight. At this moment, my job duties have changed, yet I still have the honor to say that I am a part, even small, for most of the SIEM and ArcSight history.

A Shared Story

ArcSight history goes almost in parallel with SIEM history. I’m no historian, but from my point of view and experience, I tend to see the SIEM history as going through different periods or generations, where I believe we are just starting the fifth one. Arguably, ArcSight is one of the pillars of this technology that it's been there in each one of them. It is important to note that the term SIEM was first officially used in a Gartner report titled "Improve IT Security with Vulnerability Management" from May 2005 authored by Amrit T. Williams and Mark Nicolett. Below is a breakdown of the SIEM history periods as I see them. Please keep in mind that the demarcation years should be considered an estimation with probably at least /- 1 year as a margin of error.

SIEM’s Evolution

1999-2005 – A New Hype: SIM vs. SEM

During this time, two different technologies were trying to break ground based on the security management needs as organizations were trying to take control of their security posture. These technologies were Security Information Management (SIM) and Security Event Management (SEM). SIM was mainly related to storage, analysis, and reporting of the data, whereas SEM was related mostly to the monitoring, correlation, and alerting. In retrospect, I would say that I used these terms interchangeably. Even though the concept of centralizing logs collection was not a novel idea during this time, one could ascertain that SEM was a relatively new idea around 1999, and arguably could've been the start of what it will become the SIEM market. I like to think that the end of this period is marked by the release of the May 2005 Gartner report. (Side note: the product that could be considered a pioneer in SEM technology is the precursor of NetIQ Sentinel which is currently converging with ArcSight).

2005-2010 – The Dawn of the SIEM

I would like to think that this period started when a Garner report officially coined the term SIEM, but if memory doesn't fail me, I believe, as far as 2007, there were some references of ArcSight as a SIM. Regardless, during this period is arguably when the first "official" SIEM platforms arrived, capable of doing both SIM and SEM functions. Granted, as SIEM started to become more commonly used, late in this period, then these functions were more associated with terms such as "log management" and "correlation engine." In my opinion, what ended up being the first key limiting factor for these solutions was data collection needs, as most of them struggled to expand horizontally (some still do!). As the SIEM market started to take form, this period is also marked by large corporations opening their checkbooks and for the first time buying their way into this market (examples: here, here and here), even though some of them ended up shelving these solutions down the road.

2010-2014 – SIEM: The Expanse

I like to think the beginning of this period was marked by the introduction of platforms that were able to scale pass those from the prior period. During this period, platforms achieving over 20 thousand events per second (EPS) became the new norm. This throughput was somehow impossible or at least extremely difficult to achieve before. Interestingly enough, the ability of increased throughput became somehow the precursor of what I like to call the next limiting factor. I used to say quite often "garbage in, garbage out" to customers during this time (I still do!). As you were able to send a lot more data to these platforms, the most common practice was to send everything to it without any forethought. This decision resulted in analysts drowning in a sea of data and becoming essentially blind to what was happening in their environments. That being said, this period is also marked by the increase of players in this market. If you look at the Gartner SIEM MQ images from this period, you could see that at one point, they evaluated up to 24 different SIEMs, averaging around 19 players during this time.

2014-2019 – The Rise of the UEBA

I like to think the beginning of this period is when a group of User and Entities Behavior Analytics (UEBA) solutions started to become available. Analysts were becoming overwhelmed, and rule-based solutions alone were not enough for the ever-changing security landscape. This realization led to the introductions of these solutions, where their premise is mainly baseline "entities" activities in your environment using mathematical models, and increase their risk when deviations of such baseline are found. It is also during this period that phrases such as "machine learning" and "artificial intelligence" were somehow overused, adding confusion. During this period, it started trending the idea that this technology would be the replacement of traditional SIEM or the Next-Gen SIEM. That being said, like any new technology, implementations turned out to be more difficult than expected. My exposure to a few of these solutions somehow led me to think that even though their backends are based on big data platforms, data consumption at high throughput was still an issue. Not only for the collection speed but also determining which data to collect and the quality of it (again "garbage in, garbage out"). Also, these solutions run their models on a scheduled basis, so in my opinion, "real-time" in these solutions appear to be more of a loose term. Another thing that started to become apparent, at least to me, is that some use cases were best addressed with traditional SIEM. Even Gartner stated in their UEBA market analysis report that this market will stop being a stand-alone one and will be embedded in other solutions.

2019-Now – The Smart SIEM Awakens

I believe we are currently in a newer period in SIEM history, where a new approach is emerging. Something along the lines of converging traditional SIEM and UEBA technologies but also adding orchestration capabilities (or integrations to) in the effort to optimize incident response and handling. These SIEM solutions must be able to scale as needed, as well as being able to deploy on-premises or in the cloud, but also include options for MSSP and SaaS. This solution must be smarter than previous ones, since now more than ever, analysts must do more with less and faster. If the solution can provide quick answers or relevant context to the 5 W's and 1 H questions, as well as to provide additional context, such as "have we seen this before?", "is this a known bad actor?" and "how was it mitigated?", then analysts can respond quicker and more effectively. It is important to note that the Smart SIEM term is not my term since I first read it in a security blog at the MFI community website titled "It SIEMs to Me…". That being said, I am also a strong believer that these solutions must be able to stand and comply with "the burden of proof" and "chain of custody." I know this applies to legal context, yet, besides helping stop an attack, I would like to think that organization might want for such "bad actors" to face the consequences of their actions in a court of law. So, it will be useless if these solutions have integrity issues, such as allowing data from being selectively modified or deleted.


I know there is a lot more to cover about Security Operations and SIEM, yet I believe I covered enough to demonstrate my point. The SIEM market has a vibrant history, where only solid solutions have survived, and ArcSight has been there since the beginning and is still here. In my opinion, those who learn from the past are better positioned to move into a successful future. ArcSight has been one of those products that no one could argue are one of the pillars of the SIEM history. Now, with the addition of NetIQ Sentinel and Interset,  as well as the other MFI solutions, it places ArcSight in a strong position into the SIEM future by providing all the components needed in a Smart SIEM.

Please join me in saying happy birthday to this incredible, longstanding solution, and may you have many more, ArcSight! Learn more about what ArcSight is doing for its Next-Gen SIEM. 


Guest post by Eugenio Marrero, CISSP, Director Sales Engineering, Federal at MFGS, Inc. 

This post originally appeared on Eugenio Marrero’s LinkedIn blog.


Security Operations