Mutilpe utility program on AIX server after upgrading the agent from 11.x to 12

After upgraing the agent on some unix servers I receive a Health notification alert,

Health notification for 12.03.003 utility: Process id for 'utility' got changed.
utility: Process id for 'utility' got changed.
utility: Process id for 'utility' got changed.
utility: Process id for 'utility' got changed.

when i check the agent status I see mutiple utility services running. 

x1kousra on deerap99:/usr/lpp/perf/bin $ sudo /usr/lpp/OV/bin/opcagt -status
utility Perf Agent analysis program (4915252)Running
utility Perf Agent analysis program (7405606)Running
utility Perf Agent analysis program (8585302)Running
utility Perf Agent analysis program (2687434)Running
utility Perf Agent analysis program (5046660)Running
utility Perf Agent analysis program (6750674)Running
utility Perf Agent analysis program (7471382)Running
midaemon Measurement Interface daemon (8650832)Running
ttd ARM registration daemon (6947018)Running
perfalarm Alarm generator (7602456)Running
perfd real time server Stopped
hpsensor HP Compute Sensor AGENT,OA (10092598) Running
oacore Operations Agent Core AGENT,OA (5374410) Running
opcacta OVO Action Agent AGENT,EA (11796546) Running
opcle OVO Logfile Encapsulator AGENT,EA (7668002) Running
opcmona OVO Monitor Agent AGENT,EA (12124254) Running
opcmsga OVO Message Agent AGENT,EA (9699356) Running
opcmsgi OVO Message Interceptor AGENT,EA (8519782) Running
ovbbccb OV Communication Broker CORE (5570862) Running
ovcd OV Control CORE (9568482) Running
ovconfd OV Config and Deploy COREXT (5308424) Running
Agent Health Status: OK, Time:Mon Mar 5 16:18:22 2018

Message Agent is not buffering.

 

I have tried to kill and restart the agent, restart the ovpa services and have also tried to kill and start ovc. 

Mutiple utility program keeps coming backup and a alert comes on the OMi screen that  their pid has changed. 

Is this a normal event? or else how do we manage these multiple utility programs? 

  • These are Performance Agent processes.

    What OA licenses have you used?

  • Only the "HP Operations OS Inst Adv SW LTU" is set to permanent.

  • How about stopping PA processes by this,

    # /opt/perf/bin/ovpa stop; /opt/perf/bin/midaemon -T; /opt/perf/bin/ttd –k

  • Hi,

    From which OA 11.?? version did you upgrade to OA 12.03?

    What are the current versions of the OA binaries, share 'perfstat -v' result.

    Cheers,

    Priya

  • Most of these upgrades were from 11.16.002 version to the 12.03 version. 

    Please find the perfstat details. 

    **********************************************************
    *** perfstat for xxxxxx on Tue Mar 13 15:37:38 CET 2018
    *** AIX xxxxx 1 7 00F812614C00
    **********************************************************
    Listing the installed perf products:
    HP GlancePlus 12.00.000
    HP Performance Agent 12.00.000

    listing version strings for performance tool files:
    --------------------------------------------------

    NOTE: The following software version information can be compared
    with the version information shown in the /usr/lpp/perf/ReleaseNotes file(s).


    Perf Agent executables in the directory /usr/lpp/perf/bin
    ttd 12.03.003 2017-04-05_1235 AIX 6.1_64
    midaemon 12.03.003 2017-04-05_1235 AIX 6.1_64
    perfalarm 12.03.003 2017-04-05_1235 AIX 6.1_64
    utility 12.03.003 2017-04-05_1235 AIX 6.1_64
    dsilog 12.03.003 2017-04-05_1235 AIX 6.1_64
    sdlcomp 12.03.003 2017-04-05_1235 AIX 6.1_64
    sdlutil 12.03.003 2017-04-05_1235 AIX 6.1_64
    agsysdb 12.03.003 2017-04-05_1235 AIX 6.1_64
    lsdaemon 12.03.003 2017-04-05_1235 AIX 6.1_64

    Perf Agent libraries in the directory /usr/lpp/perf/lib64
    libscope.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libnums.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libarm.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libarmNOP.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    liblicense.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libarmns.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libOvDirectAccess.so 12.03.003 2017-04-05_1235 AIX 6.1_64

    Perf Agent 32-bit arm libraries in the directory /usr/lpp/perf/lib
    libarm.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libarmNOP.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libarmns.a 12.03.003 2017-04-05_1235 AIX 6.1_64

    Perf Agent metric description file in the directory /usr/lpp/perf/lib64
    metdesc 12.03.003 2017-04-05_1235

    Perf Agent libraries in the directory /usr/lpp/OV/lib64
    libOvScopeAccess.so 12.03.003 2017-04-05_1235 AIX 6.1_64

    All critical Perf Agent files are accessible

    Perf Agent coda executables in the directories /usr/lpp/OV/bin /usr/lpp/OV/lbin /perf
    ovcodautil 12.03.003 AIX 6.1
    oacore 12.03.003 AIX 6.1
    ovbbccb 12.03.003 AIX 6.1

    Perf Agent coda/BBC libraries in the directory /usr/lpp/OV/lib64
    libOvBbc.so 12.03.003 AIX 6.1
    libOvCoda.so 12.03.003 AIX 6.1
    libOvXpl.so 12.03.003 AIX 6.1

    GlancePlus executables in directory /usr/lpp/perf/bin
    glance 12.03.003 2017-04-05_1235 AIX 6.1_64
    xglance 12.03.003 2017-04-05_1235 AIX 6.1_64
    perfd 12.03.003 2017-04-05_1235 AIX 6.1_64
    padv 12.03.003 2017-04-05_1235 AIX 6.1_64
    mpadv 12.03.003 2017-04-05_1235 AIX 6.1_64
    cpsh 12.03.003 2017-04-05_1235 AIX 6.1_64

    Glanceplus libraries in the directory /usr/lpp/perf/lib64
    libperfdaccess.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libcmdline.a 12.03.003 2017-04-05_1235 AIX 6.1_64
    libperfdadv.a 12.03.003 2017-04-05_1235 AIX 6.1_64

    ComputeSensor executables in directory /usr/lpp/OV/hpcs
    hpsensor 12.03.003 AIX 6.1

    Operations Agent in the directory /usr/lpp/OV/bin
    Operations Agent 12.03.003
    managed by OVO on xxxx

    Perf Agent Datacomm Type: http

    ************* (end of perfstat -v output) ****************

     

    There are a mix of some Linux and AIX server having the same issue.

    I received alert every time I acknowledge them that the Health notification for 12.03.003 utility: Process id for 'utility' got changed. utility: Process id for 'utility' got changed.