Add detection of flapping interface

Idea ID 2694546

Add detection of flapping interface

Our TASK:

We need to get an incident for a flapping interface, be it a single or several flaps.

What we tried:

We searched for something out-of-the-box, without luck, nor could we find any (limitedly convoluted) way to reliably detect a flapping interface.
This is what we then tried:

  1. Created a new Management Event to indicate a flapping interface
  2. Created new Causal Rule in Custom Correlation Configuration with

    the incident (item 1.) as 'Parent Incident' and

...when we tried

   Common Child Incident Attribute:  ${valueOfCia(\Q.1.3.6.1.2.1.2.2.1.1.\E.*)}  (which is ifIndex)

   Result: A linkDown trap for interface Gi1 in router-1 got correlated to a linkUp trap for interface Gi1 in router-2  (which isn't a flapping interface).

...when we tried

   Common Child Incident Attribute:  ${valueOfCia(\Q.1.3.6.1.4.1.11.2.17.2.2.0\E)}}  (which contains IP/hostname)

   Result: A linkDown trap for interface Gi1 in routerA got correlated to a linkUp trap for interface Eth2 in routerA (which isn't a flapping interface either).

 

...and it seems to be not possible to configure a Common Child Incident Attribute with

 the two required conditions, a match of ifIndex AND IP/hostname, at the same time.

 

When we searched for flapping interfaces and how to detect them, we found some really old KMs, all seemingly still looking to be served.

Also, we found a more generic Idea, https://community.microfocus.com/t5/NOM-Idea-Exchange/Better-SNMP-Trap-Rules-in-NNMi/idi-p/1644045  - in 'Status: Waiting for Votes', thus not yet ready for use.

Same for https://community.microfocus.com/t5/NOM-Idea-Exchange/Pairwise-correlation-enhancement/idi-p/1794074 which seems to be related, too.

 

So our question is:

How can we detect a flapping interface and get an incident for that? Above attempt is clearly useless so far.

We just cannot believe that there is nothing for such a basic task?! Please let us know how this can be achieved - ...or please do get this added.

7 Comments
Micro Focus Expert
Micro Focus Expert

Dear Submitter,

Thanks for the interesting Idea. I wanted to check if we can make use of Rate based correlation to achieve this?

Micro Focus Expert
Micro Focus Expert

Dear Akash,

thank you for your comment.

Yes, if you mean configuring a Rate on SNMPLinkDown (.1.3.6.1.6.3.1.1.5.3) with  Count:2 , Minutes:1 , ComparisonCriteria:Name SourceNode SourceObject

That works great for catching/notifying 2 or more flaps, and it clearly points to the affected interface and node,

however, it is left unclear if the interface finally is down or up, and,
sadly, it does not work for catching a _single_ flap

- and even a single flap may require notifying operators to get a certain service back to operational.

Micro Focus Expert
Micro Focus Expert
Status changed to: Waiting for Votes

The idea has received an initial review to ensure adherence to our idea submission and community guidelines. More information may be needed at this stage, and we expect the community to help prioritize the idea with comments and community support (votes/kudos).

Cadet 3rd Class Cadet 3rd Class
Cadet 3rd Class

The network operator ask directly for interface flaps. If not signaled these short time outages are hard to find.

The NNMi would dampen such short lasting outages (flaps) out of the box, but these little outages may severely affect the transport services in our MPLS backbone.

They may also give an indication that a line card or an interface is at its end of live and should be replaced.

 

Captain
Captain

Flapping interfaces are certainly a concern.  We've addressed this inelegantly by utilizing bgpBackwardTransitions traps with an enrichment.  We set the transition trap to a warning severity and then look for a transition from 6 to 1.  That event gets enriched to a critical severity.  

It doesn't help for a direct interface flap, but when it comes to our WAN connectivity this seems to give us what we need.  

Again, not pretty, but it does the job.  

 

Captain
Captain

@AkashDeep where are we on review and vote status?

Micro Focus Expert
Micro Focus Expert
Status changed to: Under Consideration
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.