New Issue with ESM 6.8 Patch2
There is an issue in a new Patch when Active List when new entries in some active lists are not being populated.
Also it seems like purging Engine suddenly stops after a while.
For example today it happened at 05:46 AM, when purging engine suddenly stopped and ALs got overloaded. The last log with "Purged AL" keyword appeared:
[2015-11-20 05:46:02,468][INFO ][default.com.arcsight.common.activelist.DefaultActiveListCache] Purged AL System - Resource ChangeLog removing 0 entries in 116 msec, new size = 390189
After that all the "Purged AL" logs has dissapeared.
Is anyone experiencing this?
UPDATE: Also have a look at screenshots that I've just taken. Looks kinda funny when playing with TTL. Clearly a bug.
Since we consider the issue you refer to as solved in 6.8c, it is interesting (and naturally disturbing) that you have identified it in 6.8c P2. It would be valuable if you report the issue to support. Feel free to pass me the ticket number so I can accelerate in the backend.
OK, here's an update. We also have this type of error everytime we are logging on to a console. I see this for the second time since Patch2 Install. Maybe there is a relation to the bug.
There was an error while logging into core services of type: class java.io.IOException.
Please consult log files to diagnose the issue.
Login will still continue, but some services may be affected.
We were the ones who reported the issue initially and havent seen it on 6.8cP2. My guess though is that support\dev is going to tell you the number of entries in your AL cache (in memory) is too high. The screenshot you had showed a capacity of 5 million entries and the log message was for 390,000 entries which I can tell you that support is going to probably tell you is too big (may not actually be the issue but they may have you try and reduce those first). Hopefully they can figure it out, otherwise you may have to cronjob a manager restart (we had to do that last time we had similar issues until we could fix it).
I think we were able to resolve this issue with HP Support by rebuilding AL cache:
/etc/init.d/arcsight_services stop manager
mv /opt/arcsight/manager/tmp/tuple/local/classes/ /opt/arcsight/manager/tmp/tuple/local/classes_old
mv /opt/arcsight/manager/tmp/tuple/local/source/ /opt/arcsight/manager/tmp/tuple/local/source_old
/etc/init.d/arcsight_services start manager
Since then we haven't experienced this issue for a week already.
Will have a look further and post additional information later.