NNMi pairwise correlation queue growth

Hi, experts

NNMi is receiving about 15-40 traps per second. It looks there are quite many "flapping" events, in particular at night time. As a result, pairwise correlation queue is growing, correlations are delayed for minutes and hours and then seems stopped.

The following records are seen in NNMi trace log:

2024-03-13 09:41:33.795 WARNING [com.hp.ov.nms.topo.spi.server.concurrent.NmsTimerTaskImpl] (NmsWorkManager Scheduler) Skipping task execution because previous execution has not completed: Pairwise Correlation Timer Task

Any suggestions what could be fine-tuning approach (on NNMi side, indeed) - reduce trap storage size (now is default 100k), fine tune Postgres settings or else ?

thank you in advance,
Gedas

  • Verified Answer

    +1  

    Hello Gedas,

    Not sure what is NNMi version but there was a Pairwise performance improvement in the following versions where Pxx is the patch number.

    NNMi 2019.05.P11
    NNMi 2020.11.P03
    NNMi 2021.05.P03
    NNMi 2021.11.P02
    NNMi 2022.05.P02
    NNMi 2022.11.P01

    Pairwise uses cache of opened incidents. If the number of opened incidents in DB is very large it may take longer to build the cache.

    Also, if you stop NNMi services trapreceiver is still running and keeping all new traps in cache. If you start NNMi let's say in 5 min all cached traps need to be processed including Pairwise. With large number of traps Pairwise may start falling behind. In general, it is difficult to provide more suggestions without knowing details on NNMi Health and sizing,

    Best regards,
    Sergey