NNM-Events Health Agent

Hello,

I got the the following event  and can't find any information about it:

Events Health Agent
[Major] Failed attempts to persist incidents 1.

Can anybody explain what to do to get back the system in "normal" state?

 

Thanjks,

CR

Tags:

  • Hi...This will explain what you are seeing and how to

     

    Error Message

    Following error can be seen in Network Node Manager (NNM 9i) system health report :

    Events Health Agent

    [Major] Failed attempts to persist incidents 1.

     

    Note : The report can be accessed from NNMi User Interface (UI) > Help > System Information > Health > View detailed health report.

     

    Cause

    The event 1 indicates that Network Node Manager (NNM 9i) tried writing the incidents into the database, but that attempt failed. This can happen when multiple users are trying to trim/archive the incidents database and Network Node Manager (NNM 9i) is also generating an incident at the same time to be written in that database. Also, this could be potential issue of incidents reaching their limit of 100k.

     

    Workaround / Fix

    Please check the number of incident generated by NNM 9i (Go to NNMi User Interface (UI) > Help>System Information>Database). If incidents has crossed the 100k limit in Network Node Manager (NNM 9i), please try to perform the trim operation using nnmtrimincidents.ovpl command.

     

    Note: For more info on nnmtrimincidents.ovpl command please go to NNMi User Interface (UI) > Help > NNMi Documentation Library>reference pages.

     

    Hope this helped...

  • n addition you may want to look at the nnmhealth.ovpl command.

     

    Thanks

     

     bin]# ./nnmhealth.ovpl
    Usage: nnmhealth -print (quiet|brief|detailed|conclusions|agents|history|verbose [-filter <agent list>]) | -suppress <conclusion list> | -activate <conclusion list>
    Arguments
      -activate <conclusion list>: Activates one or more suppressed conclusions.
      -filter <agent list>: Filter information with agent names.
      -print: Reads the current status of NNMi
      -suppress <conclusion list>: Suppresses one or more active conclusions. This will take effect until NNMi is restarted or until the conclusion is re-activated.
    Additional Arguments
      -jndiHost <hostname>: The server jndi host; default is localhost
      -jndiPort <port>: The server jndi port; default is 1099
      -p <password>: The password of the user
      -u <username>: The username to run this command

  • HI,

     

    NNM JBOSS subservices is not running.please see the ovstatus output

     

     


    C:\Documents and Settings\hptadmin>ovstatus -v ovjboss
     object manager name: ovjboss
     state:               RUNNING
     PID:                 7272
     last message:        Initialization complete.
     exit status:         -
     additional info:
                SERVICE                                 STATUS
                CPListener                              Service is started
                CommunicationModelService               Service is started
                CommunicationParametersStatsService     Service is started
                CustomPoller                            Service is started
                DatabaseMaintenance                     Service is started
                EventsCustomExportService               Service is started
                ExtensionDeployer                       Service is started
                HealthAgent                             Service is started
                InstanceDiscoveryService                Service is started
                IslandSpotterService                    Service is started
                KeyManager                              Service is started
                LDAP                                    Service is started
                LWSSO                                   Service is started
                LicensingHealthAgent                    Service is started
                ManagedNodeLicenseManager               Service is started
                ModelChangeNotificationAdapter          Service is started
                ModelChangeRegister                     Service is started
                MonitoringSettingsService               Service is started
                NMSLogManager                           Service is started
                NamedPoll                               Service is in created state
                NetworkApplication                      Service is started
                NmsApa                                  Service is started
                NmsCustomCorrelation                    Service is started
                NmsDisco                                Service is started
                NmsEvents                               Service has failed. Refer to the com.hp.ov.nms.events.NmsEvents log file for details.
                NmsEventsConfiguration                  Service is started
                NmsExtensionNotificationService         Service is started
                NmsModel                                Service is started
                NmsWorkManager                          Service is started
                NnmTrapService                          Service is started
                PerformanceSpiConsumptionManager        Service is started
                PolicySynchronizer                      Service is started
                RbaConfig                               Service is started
                RbaManager                              Service is started
                SpmdjbossStart                          Service is started
                StagedIcmp                              Service is started
                StagedSnmp                              Service is started
                StatePoller                             Exception on service: com.hp.ov.nms.statepoller.common.StatePollerException: Failed to load StatePoller policies fro
                SystemMonitor                           Service is started
                TrustManager                            Service is started

  • Hi,

     

     I would be glad to take a look at this for you. Please zip/tar all files in the NNMDATADIR/log/nnm/* directory and attach the file to this message. I will analyze the files and let you know why the jboss processes are not running properly.

     

    Thanks

  • Thanks for the reply

     

     

     

     

    i am attaching all the log file .

    nnm.rar
  • I am seeing there are some database problems. Please send me the following output of these commands. Thanks
    -ovstatus -c
    -nnmversion.ovpl
  •  I am seeing there are some database problems. Please send me the following output of these commands. Thanks
    -ovstatus -c
    -nnmversion.ovpl



  • C:\Documents and Settings\hptadmin>ovstatus -c
     Name                   PID  State          Last Message(s)
     OVsPMD                8144  RUNNING        -
     pmd                   8532  RUNNING        Initialization complete.
     nnmaction             7864  RUNNING        Initialization complete.
     nmsdbmgr             10480  RUNNING        Database available.
     ovjboss               5492  RUNNING        Initialization complete.

    C:\Documents and Settings\hptadmin>nnmversion.ovpl
    NNM 9.00,9.01.002,9.01.004,9.01.005
    NNM Patch: NNM900_00005

  • HI,

     

    I am not able to take nnmbackup

     

    C:\Documents and Settings\hptadmin>nnmbakup.ovpl -force -target C:\nnmbackup5-09
    -12\new backup
    'nnmbakup.ovpl' is not recognized as an internal or external command,
    operable program or batch file.

    C:\Documents and Settings\hptadmin>nnmbackup.ovpl -force -target C:\nnmbackup5-0
    9-12\new backup
    Skipping empty property definition: topology.fs in file D:/NNM9i_DATA/shared/nnm
    /backup.properties.

    Found special file placement key embdb.dir=<OV_DATA>/shared/nnm/databases/Postgr
    es

    Found special file placement key hostspecific.files=<OV_DATA>/conf/nnm/props,<OV
    _DATA>/shared/nnm/conf/lwssofmconf.xml

    Found special file placement key certmerge.files=<OV_DATA>/shared/nnm/certificat
    es/nnm.keystore,<OV_DATA>/shared/nnm/certificates/nnm.truststore

    Found special file placement key licensing.files=<OV_LIC_DATA>/data/LicFile.txt,
    <OV_DATA>/shared/nnm/conf/licensing,<OV>/misc/nms/lic

    Found special file placement key versioning.files=<OV_DATA>/NNMVersionInfo

    Attempting to backup C:/Program Files (x86)/Common Files/Hewlett-Packard/HPOvLIC
    /data/LicFile.txt to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/licData/data
    /LicFile.txt

    Attempting to backup D:/NNM9i_APP/misc/nms/lic to C:/nnmbackup5-09-12/new/nnm-ba
    k-20120906114846/misc/nms/lic

    Attempting to backup D:/NNM9i_DATA/shared/nnm/conf/licensing to C:/nnmbackup5-09
    -12/new/nnm-bak-20120906114846/data/shared/nnm/conf/licensing

    Attempting to backup C:/Program Files (x86)/Common Files/Hewlett-Packard/HPOvLIC
    /data/LicFile.txt to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/licData/data
    /LicFile.txt

    Attempting to backup D:/NNM9i_DATA/shared/nnm/certificates to C:/nnmbackup5-09-1
    2/new/nnm-bak-20120906114846/data/shared/nnm/certificates

    Attempting to backup D:/NNM9i_APP/conf to C:/nnmbackup5-09-12/new/nnm-bak-201209
    06114846/conf

    Attempting to backup D:/NNM9i_DATA/conf to C:/nnmbackup5-09-12/new/nnm-bak-20120
    906114846/data/conf

    Attempting to backup D:/NNM9i_DATA/shared/nnm/conf to C:/nnmbackup5-09-12/new/nn
    m-bak-20120906114846/data/shared/nnm/conf

    Attempting to backup D:/NNM9i_DATA/shared/nnm/lrf to C:/nnmbackup5-09-12/new/nnm
    -bak-20120906114846/data/shared/nnm/lrf

    Attempting to backup D:/NNM9i_DATA/NNMVersionInfo to C:/nnmbackup5-09-12/new/nnm
    -bak-20120906114846/data/NNMVersionInfo

    Attempting to backup D:/NNM9i_APP/misc/nnm to C:/nnmbackup5-09-12/new/nnm-bak-20
    120906114846/misc/nnm

    Attempting to backup D:/NNM9i_APP/newconfig to C:/nnmbackup5-09-12/new/nnm-bak-2
    0120906114846/newconfig

    Attempting to backup D:/NNM9i_APP/nonOV/jboss/nms/server/nms/conf to C:/nnmbacku
    p5-09-12/new/nnm-bak-20120906114846/nonOV/jboss/nms/server/nms/conf

    Attempting to backup D:/NNM9i_APP/nonOV/jboss/nms/server/nms/deploy to C:/nnmbac
    kup5-09-12/new/nnm-bak-20120906114846/nonOV/jboss/nms/server/nms/deploy

    Attempting to backup D:/NNM9i_DATA/shared/nnm/user-snmp-mibs to C:/nnmbackup5-09
    -12/new/nnm-bak-20120906114846/data/shared/nnm/user-snmp-mibs

    Attempting to backup D:/NNM9i_DATA/shared/nnm/conf/props to C:/nnmbackup5-09-12/
    new/nnm-bak-20120906114846/data/shared/nnm/conf/props

    Attempting to backup D:/NNM9i_DATA/conf/nnm/props to C:/nnmbackup5-09-12/new/nnm
    -bak-20120906114846/data/conf/nnm/props

    Attempting to backup D:/NNM9i_DATA/shared/nnm/actions to C:/nnmbackup5-09-12/new
    /nnm-bak-20120906114846/data/shared/nnm/actions

    Attempting to backup D:/NNM9i_DATA/shared/nnm/www/htdocs/images to C:/nnmbackup5
    -09-12/new/nnm-bak-20120906114846/data/shared/nnm/www/htdocs/images

    Attempting to backup S:/NNM/dataDir/log/nnm/nnm/signin.0.0.log to C:/nnmbackup5-
    09-12/new/nnm-bak-20120906114846/data/log/nnm/signin.0.0.log

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/shared/nnm/certificat
    es/nnm.keystore to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/
    cert_merge/data/shared/nnm/certificates/nnm.keystore for certification merge dur
    ing restore

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/shared/nnm/certificat
    es/nnm.truststore to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/special_file
    s/cert_merge/data/shared/nnm/certificates/nnm.truststore for certification merge
     during restore

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/conf/nnm/props to C:/
    nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/local_files/data/conf/
    nnm/props for local system restore

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/shared/nnm/conf/lwsso
    fmconf.xml to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/local
    _files/data/shared/nnm/conf/lwssofmconf.xml for local system restore

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/licData/data/LicFile.txt t
    o C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/licensing_files/l
    icData/data/LicFile.txt for licensing restoration

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/shared/nnm/conf/licen
    sing to C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/licensing_f
    iles/data/shared/nnm/conf/licensing for licensing restoration

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/misc/nms/lic to C:/nnmback
    up5-09-12/new/nnm-bak-20120906114846/special_files/licensing_files/misc/nms/lic
    for licensing restoration

    Moving C:/nnmbackup5-09-12/new/nnm-bak-20120906114846/data/NNMVersionInfo to C:/
    nnmbackup5-09-12/new/nnm-bak-20120906114846/special_files/versioning_files/data/
    NNMVersionInfo for versioning restoration

    Problems encountered while backing up database - please review the following
    possible issues:
     pg_dump: [archiver] WARNING: requested compression not available in this instal
    lation -- archive will be uncompressed
    pg_dump: Dumping the contents of table "jbm_msg" failed: PQgetCopyData() failed.

    pg_dump: Error message from server: server closed the connection unexpectedly
            This probably means the server terminated abnormally
            before or while processing the request.
    pg_dump: The command was: COPY public.jbm_msg (message_id, reliable, expiration,
     "timestamp", priority, "type", headers, payload) TO stdout;


    Backup operation failed.

  • Please install the attached hot fix which should resolve any database issues you are having. Let us try to fix this first.

    If you full backup does not work you can use nnmconfigexport.ovpl to backup your configuration and nnmbackupembdb.ovpl to back up your current database.

     

    EX: nnmconfigexport.ovpl -c all -f \TMP

            nnmbackupembdb.ovpl -force -target \NNMDBBACKUP

    HF-NNMI-9.0XP5-POSTGRES-WINDOWS-20120803.zip