Having problems with your account or logging in?
A lot of changes are happening in the community right now. Some may affect you. READ MORE HERE

Improve message Event Storm Suppression to be more like OML's ESF

Improve message Event Storm Suppression to be more like OML's ESF

The current Event Storm Suppression in OBM/OMi is much less versatile than the Event Suppression Filter (ESF) feature in the legacy OML 9.2X.

Before 2018.11, the Event Storm Suppression in OBM only allowed to suppress storms of events coming from the same source node when they exceeded a rate.

In OBM 2018.11, the Event Storm Suppression feature has been improved to, in addition, allow to suppress storms of events with the same related CI. This is a very useful improvement. For example, when the events come from an NNMi system, end network devices are the ones that will most be sending storms, but all the events come from a single node, the NNMi system. This change helps in this situation, since we do not want to block the whole NNMi system, only the end network devices that generate too many events.

But still, the Event Storm Suppression is not versatile enough. For example, it is common to deploy a buggy policy to hundreds of nodes and suddenly getting thousands of events from thousands of nodes, and Event Storm Suppression would not detect this because the events come from different nodes and even different related CIs. Also, for example, it could happen that all the Linux systems get a kernel patch, that kernel patch generates 20 several events in syslog and a policy to match kernel problems in all your thousands of nodes would generate an important event storm without a common node or related CI.

In the legacy OM world, the Event Suppression Filter (ESF) provided a very fine storm control. It was possible to define "gates" to define how a given storm should be detected. This is a sample "gate":

GateName=Network
Customer=!
Node=%
Object=%
Application=!
MsgGroup=Network
MsgType=!
MsgText=%
TT=1
Rate=7
Period=5
Annotate=0
Log=1
CreateTT=1
CloseMsg=1
CloseAction=none

We can see that we could define that the Event Storm filter for "Network" would be applied on events with the same Customer, Application and MsgType and with the "MsgGroup=Network". But we could also use other fields (Node, Object, Message Text) to define what we would consider an Event Storm and how to react to it.

I would like to see a feature as rich as OML's ESF in OBM that would allow us to categorize types of potential Event Storms and how to react to them. I would specially would like to see the capability to categorize event storms per source policy and category, but if possible, we should have additional fields to choose, like with OM's ESF.

4 Comments
Mel Farber Super Contributor.
Super Contributor.

Our event storm implementation doesn't exactly address the situation described, but useful for others as it has been for us.

We created a stream based event filter.  If we see X events, related based on an attribute we create with a groovy script, in Y minutes we create a new event and release all the events.  We could close the events and someone smarter than me might figure out a way to suppress the events causing the storm.

We are very tight in our policies so we rarely get storms, except during network outages.  Everyone's situation is different, but I thought I would share the approach we use.

Mel

Micro Focus Expert
Micro Focus Expert

tfewshe

Micro Focus Expert
Micro Focus Expert

Hi Vicente,

Thanks for providing the detailed description in the idea. Have you looked at the Event Suppression Rules in OBM, they provide you capability to define many other attributes in the event filter that can be used to suppress specific events.

Please take a look and share your comments here.

Thanks & Regards,

Moderator, OpsB IdeaX

Micro Focus Expert
Micro Focus Expert
Status changed to: Waiting for Votes

Hi Vicente, and eveyone who has voted,

We would like to get your feedback on options like SBEC and event suppression rules as an alternative before this is considered further.

Thanks & Regards,

Moderator, OpsB  IdeaX

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.