Highlighted
Absent Member.
Absent Member.
1360 views

What is the absolute EPS limit for ESM?

Anyone know?  I know there are many factors involved, but just as you can safely say no human can run faster than 40MPH, there has to be a limit you can safely say ESM will not be able to exceed.

Thanks!

Joe

Labels (1)
0 Likes
Reply
22 Replies
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Re: What is the absolute EPS limit for ESM?

Hi Joe,

1st theory : As the logger is supposed to be the fastest solution, we can consider it's theoretical max eps value as being the max for ESM too. ( 100.000 eps if I remember correctly )

2nd theory : we can think about it differently.  As you are a specialist in high throughput ESM ( I even suspect you to use liquid nitrogen to boost your EPS rate 😉 , where is, according to you, the bottleneck in your fastest ESM implementation ?  If it is the CPU, measure the theoretical performance gap between the CPU power on that system and the most powerful system available on the market and start making assumption about performance you would get with that hardware.

3rd theory : choose an absolute technical limitation that you are sure about like the number of IOPS you can get on the fastest storage in the world ( AFAIK it's 5 millions IOPS and this is not a typo ).  Now consider that your performance would increase linearly compared to your current iops and calculate the result you might expect.  This is purely theoretical as performance don't increase linearly and you might have another bottleneck before reaching the iops limit but it will surely give you the maximum EPS you will never be able to outperform

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

I believe it's actually a software limitation.  Something to do with threads.

I'm getting 13,000 EPS on some of our systems, but I've heard some ArcSight folks quote ESM as topping out around 8,000.  Perhaps Stefan, or someone from ArcSight could elaborate.

PS: No one needs to know about my liquid nitrogen experiments..  😉

-Joe

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

What are the hardware specs for the 13K systems?
0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

The hardware specs for 13k EPS isn't much in my opinion:

  • Manager and DB are both 8 core Xeon with 10GB of RAM
  • 10k RPM fibre Ch drives in a SAN with 4GB fibre

server.properties mods:

  • servletcontainer.jetty311.threadpool.maximum=256
  • agents.threads.max=128

bleep mods:

  • -Xms2048m -Xmx4096m

-Joe

0 Likes
Reply
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Re: What is the absolute EPS limit for ESM?

Joe,

Are you running reports/trends on the 13K eps system? Are these daily or hourly reports/trends?

Thanks,

Doron

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

There are some rules, active channels etc, but really the only thing querying the database is a single active channel while Bleep is running.
0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

We have system running 5-6k EPS with full set of rules, channels, trends. based on 2 servers with 2Quads X 10-12GB RAM with external SAS storage (SAS cable, not fiber)
0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

It varies with the ArcSight setup.

Before starting with the installation you must have calculated the EPS and choose the setup accordingly.

If not make sure you are within your EPS limit.

Thanks,

Vivek

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

I'm actually referring to the limitations of the ESM software, not the limitations of currently available hardware.

Thanks!

-Joe

0 Likes
Reply
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Re: What is the absolute EPS limit for ESM?

Hi Vladi,

Can you provide some insights and tips as to how reports and trends

should be written for such system? E.g. use only indexed fields, etc.

Thanks,

Doron

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: What is the absolute EPS limit for ESM?

Sure

1. Reports should use only active lists or trends.

2. Trends should collects data from active lists (patch 3 for 4.5 sp1 solve some timestamp issues). We creates rules that fill data to active lists for short period of time and then trend poll data from those active lists

vladi

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.