Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Have you checked to see if the "Rules Engine" section is still present after restarting it? I tried it on two different systems and the section is no longer there after the first rules engine restart, making only 1 possible restart between each manager restarts.

Also, The data monitors are also failing on my dev box after the rules engine restart...

0 Likes
Established Member.. raymond.doty
Established Member..

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Answer, thanks for pointing this out.  It seems the same thing happened to us as well.

The following data monitors have died:

-Database inserts/retrievals (ArcSight side)

-Rule top firing/partial matches

-Rules engine internal stats

-All moving average data monitors

Restarting them has not brought them back online.  I am going to have to restart the manager, because we use some moving average for critical monitoring functions.

I guess the promising bit out of all this is that it is either related to the above content or the rules engine...

0 Likes
Established Member.. raymond.doty
Established Member..

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Haha, we were trying to figure that out last night. Yes, it seem that after you restart the rules engine, the rules engine disappears from the manage.jsp page...

I'm thinking this is not a solution, given the issues... I'm going to strikethrough the post above to make sure people done use it.

0 Likes
jbur Absent Member.
Absent Member.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Still the closest thing to a fix we've had in weeks!  No free lunch I guess.  Might be just the thing for folks that don't use rules or dashboards...   😉

-Joe

0 Likes
Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution


About the same data monitors on my side.

Also, in the server.log, lots of "MISSING MONITOR EVENT" entries...

0 Likes
Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Another BIG word of caution, it seems that it also prevents the active lists from being updated! As soon as I did the engine restart this morning, all the active lists stopped being updated, only had "entry expired" events. I restarted the whole manager about 30 minutes ago, and the AL are now updating as they should.

So this somehow proves that there is an underlying problem with the rule engine causing the caching, but restarting the rules engine is pretty buggy also...

0 Likes
Established Member.. raymond.doty
Established Member..

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Alright, well - we will consider that idea scratched.  I knocked it out above...  Yuck, we are restarting our production environment.  Going to go back to support and tell them about all these breakages.

YUCK!

0 Likes
Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

After a bit more investigating, I found that the "restart rules engine" thing actually stops the rules engine, and it doesn't come back up. I had ZERO correlation events between the rules engine restart and the actual manager restart. This is why the dashboards and active list weren't updating, the needed update events never actually appeared.

So another bug to add to the list!

0 Likes
jbur Absent Member.
Absent Member.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Answer wrote:

After a bit more investigating, I found that the "restart rules engine" thing actually stops the rules engine, and it doesn't come back up. I had ZERO correlation events between the rules engine restart and the actual manager restart. This is why the dashboards and active list weren't updating, the needed update events never actually appeared.

So another bug to add to the list!

This could actually be a good thing.  If someone can figure out a way to get the rules engine started again then you've got a workable solution.

-J

0 Likes
Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Workable Band-Aid, would be better if the root problem could be fixed!

0 Likes
Answer Honored Contributor.
Honored Contributor.

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Anybody got something new with support or anything?

I'm still having the content blamed for the problem... My case was apparently escalated and linked to a watch list, but still no candy.

0 Likes
Established Member.. raymond.doty
Established Member..

Re: ESM6 ingest first indication of something else bad..?

Jump to solution

Nothing of significance yet.  They have us testing various things, no success yet...

They have finally acknowledged that they are seeing it in other customers and that it is a 'high priority issue' that they are troubleshooting in multiple environments.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.