Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..
485 views

NNMi System Health (Few alerts I have never seen)

Jump to solution

Greetings,

Came accross these alerts this morning,  anyone have a suggestion on what to look for and how to resolve,  Thanks

  Pipeline output queue size exceeded higher limit 71,113. Dropping non-management event incidents.

 

The address jms.topic.nms.apa.monitoredStateChange is consuming 10,150 MB in 1,015 page files

0 Likes
1 Solution

Accepted Solutions
Highlighted
Outstanding Contributor.. Outstanding Contributor..
Outstanding Contributor..

Re: NNMi System Health (Few alerts I have never seen)

Jump to solution

Hi, David
yes, it coule be that hornerQ will be gradually proceswsed and decreased, but as far as I experence it can take long time. In particular if many files from 11 Jna are there.

Look to latest SNMP traps in e.g Custom Incidents. . What is time difference between "Origin ocurence time" and "Created time" ?  If it is gradually decresing for new created traps - then horneq is reducing. In time is more than 10-30  minutes, it's better to remove "cork"  from pipeline.

regards,
Gedas

View solution in original post

0 Likes
4 Replies
Highlighted
Outstanding Contributor.. Outstanding Contributor..
Outstanding Contributor..

Re: NNMi System Health (Few alerts I have never seen)

Jump to solution

Hi, David

have you got  SNMP traps storm recently? Any info in trapanalytics logs?
Is there some bootleneck (CPU or  disk I/O).
Are you running NNMi on VM? What is your NNMi version/patch level?  Are you getting SNMPv3 trap from environment?

Quick and dirty approach could be to delete event queue, but I think issue must be investigated - why it has happened and prevent re-occuring.

ovstop; service nettrap stop
rm -rf /var/opt/OV/nmsas/NNM/data/hornetq/*
rm -rf /var/opt/OV/nmsas/NNM/data/tx-object-store/*
ovstart


regards,
Gedas

 

Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: NNMi System Health (Few alerts I have never seen)

Jump to solution

Gedas,

I did find a device that had a recent trap rate of 175,000.  I ran a trapdump of the IP and added a couple of OID's to the trapfilter.conf file and will monitor

I also checked paging directory in hornetq and have hundreds of paging files going back to Jan 11.  Curious if those will clear by themselves.

I am running NNMi version 1000_005 on RHEL 6

Thanks

 

0 Likes
Highlighted
Outstanding Contributor.. Outstanding Contributor..
Outstanding Contributor..

Re: NNMi System Health (Few alerts I have never seen)

Jump to solution

Hi, David
yes, it coule be that hornerQ will be gradually proceswsed and decreased, but as far as I experence it can take long time. In particular if many files from 11 Jna are there.

Look to latest SNMP traps in e.g Custom Incidents. . What is time difference between "Origin ocurence time" and "Created time" ?  If it is gradually decresing for new created traps - then horneq is reducing. In time is more than 10-30  minutes, it's better to remove "cork"  from pipeline.

regards,
Gedas

View solution in original post

0 Likes
Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: NNMi System Health (Few alerts I have never seen)

Jump to solution

Gedas,

 

I stopped the process and removed the directories,  will monitor once back online,  thanks for the help

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.