ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins. Read more for important details.
ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins.Read more for important details.
Commodore Commodore
Commodore
1528 views

ESM 7.2 slowness? Anyone running ESM 7.2?

All,

Recently upgraded to ESM 7.2 on new hardware and this was after running on 7.0 patch 2 for a while on same system.  Compared to my older ESM 6.8 patch 4 systems running on older hardware: the new versions 7.2 and 7.0 Patch 2 are noticeably slower when performing searches and running reports.  The new  ESM 7.2 console also doesn’t seem to respond as well and I've even had it hang up a few times.  I know "slow" can be very subjective, but has anyone else noticed this?  Anyone have any suggestions on things to check to speed up things?  Is there a java memory setting for the console like the server java memory settings?

Thanks,

Eric

 

 

Labels (1)
0 Likes
14 Replies
Commodore Commodore
Commodore

Just an addtional note on this.  The older 6.8 and newer 7.2 ESM systems are recieving the same logs and using pretty much the same rules and other content.  

0 Likes
Lieutenant
Lieutenant

Hi ejsimon,

Did you find any solution for this? I am facing the same issue.
0 Likes
Commodore Commodore
Commodore

No, we didn't get an answer for it and have been living with it.  It's not "unusable" slow but after moving to new and faster hardware I was hoping it would perform better.  I would say that it's slightly (but noticeably) slower than our 6.8 instance with day to day searches from the ESM client.     

0 Likes
Vice Admiral
Vice Admiral

Hi All,

 

I have exactly the same issue, my legacy EMSs are performing much better than my ESM DCs running 7.3, 

no idea why, the HW for the clusters is better than the monolithic one, the only difference is the SW, not events nor content.

 

Did anyone discover the root cause behind?

 

BR

0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

We run console 7.2.1 and i also noticed some unstability in the first days.

We were using the console.exe in bin folder. Since we use console.bat (in same folder) stability seemd to have increased.

Also we were talking about the mem parameter in console.bat of bin/scripts.

not sure if i changed here anything, however this is my current conf

set ARCSIGHT_JVM_OPTIONS=-Xms64m -Xmx512m -XX:-UseThreadPriorities -XX:+HeapDumpOnOutOfMemoryError -Dsun.java2d.noddraw=true -Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true -Dsun.locale.formatasdefault=true -javaagent:%ARCSIGHT_HOME%\lib\agenthelper-1.1.jar

 

We also noticed some lowness, however i assume the reason for this is me working via VPN.

My next design for enhancing the ESM environment will be a dedicated server next to the esm, to run the console on, to avoid NW latency.

If you use the "logger interface" on ESM, you will notice, it is way faster then the fat client (yes it also uses different mechanism to fetch events from DB)

 

KR

A

Vice Admiral
Vice Admiral

Hey @vitz1 ,

 

Thanks for your inputs, but it's not the case.

We have the ArcSight console's heap set to 1gb and it's published through Citrix, is the way that we elected to provide fault tolerance but also let us share the client while using SAML 2.0

 

Regards,

 

Karl.

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

hi @Karl2 ,

Is this a distributed install? Are you experiencing slowness with a distributed install vs a compact install?


One thing to note, that i don't think ( i could be wrong) was included in previous documentation is where the information repo is placed on each node in a distributed install.

 

Placing Information Repository Instances on a Separate Partition

In a distributed correlation environment, running an information repository instance on the disk partition that contains/opt/arcsightleads to performance problems. To avoid these problems, you must create/var/opt/arcsight(as a directory or a symbolic link to a directory) on all of the cluster nodes before you install ESM. During installation, the installation program places repository data in the partition that contains/var/opt/arcsightif it exists. Otherwise, it places repository data in the partition that contains/opt/arcsight

Full documentation available here. Arcsight 7.4 Installation Guide

I haven't tried fixing this post-install, but i'd imagine you could relocated the repo to another partition and then create a symlink to /var/opt/arcsight in the original repo directory. But proceed with caution. Do not perform this on a production system. Re-installing and then importing your system tables from the previous version may be a better solution.
If in doubt, log a support ticket.

Let me know how you go,

Thanks

 

Lewis

Vice Admiral
Vice Admiral

Hey @LewisJ ,

 

Thanks for highlighting that topic, in fact, we followed the installation guides by heart :

[arcsight@socdap-pp-arcsightcl-ps ~]$ ls -larth /var/opt/arcsight/data/repo1/
total 0
drwxrwxr-x. 3 arcsight arcsight 19 Feb 9 2020 ..
drwxrwxr-x. 3 arcsight arcsight 23 Feb 9 2020 log
drwxrwxr-x. 3 arcsight arcsight 35 Feb 9 2020 data
drwxrwxr-x. 4 arcsight arcsight 29 Feb 9 2020 .
[arcsight@socdap-pp-arcsightcl-ps ~]$

 

This cluster comes from 7.0 and has been upgraded 'till 7.3.

The thing here is that when we run a search on both ESM DCs Clusters searches takes hours to complete, while our legacy clusters running compact mode, searches by far, faster: 30'. This has been detected during the last upgrade. To make things even worst, the HW were the ESM DCs runs is newer, is larger and now it's a "cluster" with great storage (Premium SSDs)

Note that arcsight esm is not a search tool, but the search times on the ESM DC are just terrible, and I have no clue why. I honestly think that it can be due to software configuration (mysql, logger), but no idea which params should be optimized, after 3M with support we haven't received any response .. we're just running out of ideas.

 

anyway, thanks a lot for the inputs shared!

 

regards,

 

karl.

0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

@Karl2 

when you worked with support:

 - i assume you tested speed via ArcSight Command Center, is this also slow?
 - support adjusted some settings in mysql config file my.cnf ?

Do you run HA as well, if so what speed are the NIC links between the HA nodes.

 

Kr.

A

0 Likes
Vice Admiral
Vice Admiral

Hey @vitz1 ,

 

 - i assume you tested speed via ArcSight Command Center, is this also slow? 

==> Correct, searches are also very slow on ACC.


 - support adjusted some settings in mysql config file my.cnf ?

==> This is why expected by the time I raised the request, no final recommendations has been given yet 😞

 

- Do you run HA as well, if so what speed are the NIC links between the HA nodes.

==> No, we assumed the risk to have the SPoF on 1 persistor instead of deploying the Active-Passive Persistor.

 

Thanks for your inputs,

 

BR,

 

Karl.

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Hi @Karl2 

 

Could you check your server.properties file and look for

search.index.level=

and see what it is set at?

Could you also check your settings in ACC under Administration -> Search -> search options and check the full-text search options and regex options?

 

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.