Highlighted
Micro Focus Expert
Micro Focus Expert
255 views

OA Support Tip: “Agent” components fail to start when running ovc -start

When starting the Operations Agent (ovc -start) all of the “Agent” components fail to start and the following error is logged to the %OvDataDir%\log\System.txt file:

 
(ctrl-41) Timeout while starting component <component name>.

 
Running the following command: "%OvInstallDir%\lbin\eaagt\opcmsga" returns the following error:

 
Internal error : Can not open message catalog .\opcagt.cat

 
This can occur if the ‘opcagt.cat’ file is missing.  

 
The ‘opcagt.cat’ file is located in the following folder:

 
%OvAgentDir%\msg\C\opcagt.cat

 
If the opcagt.cat file is missing, it must be restored.  Reinstalling the Operations Agent will resolve the issue or the file can be copied from another system with the same agent version.

Tags (1)
0 Likes
14 Replies
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hello Stella,

 

I have the same problem on one of LINUX machine and in system.txt see error exactly you mentioned.

 

0: ERR: Tue Jun 11 11:37:01 2013: ovcd (25095/1124215104): (ctrl-41) Timeout whi
le starting component 'opcmsgi'.

 

But when i execute "/opt/OV/lbin/eaagt/opcmsga" i get different error.

 

------ Error occurred during the initialization of ITO.

 

Same solution apply here?

 

-KAKA-

 

0 Likes
Highlighted
New Member.

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi Kaka

 

Looks like semaphore limit is reached on the system

 

If semaphore has indeed reached the limit please try rebooting the system

 

 

Regards

Sandeep

Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hello Sandeep,

Thank You for your response. How can i check if semaphore has reached the limit?

Asking for a reboot to UNIX team, need justification.

-KAKA-
0 Likes
Highlighted
New Member.

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi

 

These commands can help you know semaphore usage

 

ipcs -a

ipcs -m

ipcs -s

 

Thanks

Sandeep

0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hello Sandeep,

 

This is what i get. Do you see any semaphore issue?

 


------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x01068730 44597248   root      664        4192       1
0x00000000 38502401   root      664        4192       1          dest
0x010686c3 38535170   root      664        4192       1
0x01068758 44695555   root      664        4192       1

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x0106869f 0          root      664        1
0x010686a0 32769      root      664        1
0x00000000 328466434  root      600        1
0x00000000 136380419  root      600        1
0x00000000 870055940  root      600        1
0x00000000 2125594629 root      600        1
0x01068743 1598685190 root      664        1
0x00000000 327687     root      600        1
0x00000000 1595441160 root      600        1
0x00000000 142147593  root      600        1
0x00000000 1594687498 root      600        1
0x00000000 2125955083 root      600        1
0x00000000 1590853644 root      600        1
0x00000000 1590886413 root      600        1
0x00000000 1591476238 root      600        1
0x00000000 876740623  root      600        1
0x00000000 2023784464 root      600        1
0x00000000 2075394065 root      600        1
0x00000000 1760985106 root      600        1
0x00000000 980647955  root      600        1
0x00000000 1267597332 root      600        1
0x00000000 1286668309 root      600        1
0x00000000 1571356694 root      600        1
0x00000000 89817111   root      600        1
0x00000000 1460699160 root      600        1
0x00000000 1658060825 root      600        1
0x00000000 1669005338 root      600        1
0x00000000 382107675  root      600        1
0x00000000 1471086620 root      600        1
0x00000000 1768849437 root      600        1
0x00000000 2070446110 root      600        1
0x00000000 1773961247 root      600        1
0x00000000 970784800  root      600        1
0x00000000 496533537  root      600        1
0x00000000 971538466  root      600        1
0x00000000 1596227619 root      600        1
0x00000000 971833380  root      600        1
0x00000000 972488741  root      600        1
0x00000000 972783654  root      600        1
0x00000000 1189937191 root      600        1
0x00000000 1189249064 root      600        1
0x00000000 1190232105 root      600        1
0x00000000 1190690858 root      600        1
0x00000000 1191182379 root      600        1
0x00000000 1191346220 root      600        1
0x00000000 1445199917 root      600        1
0x00000000 1444708398 root      600        1
0x00000000 1445593135 root      600        1
0x00000000 1445855280 root      600        1
0x00000000 1446543409 root      600        1
0x00000000 1446805554 root      600        1
0x00000000 1577058355 root      600        1
0x00000000 1576763444 root      600        1
0x00000000 1577189429 root      600        1
0x00000000 1577320502 root      600        1
0x00000000 1577615415 root      600        1
0x00000000 1577746488 root      600        1
0x00000000 1595572281 root      600        1
0x00000000 1596653626 root      600        1
0x00000000 1591935035 root      600        1
0x00000000 1593114684 root      600        1
0x00000000 1593540669 root      600        1
0x00000000 1594753086 root      600        1
0x00000000 870318144  root      600        1
0x00000000 1597702209 root      600        1
0x00000000 142278722  root      600        1
0x00000000 141328451  root      600        1
0x00000000 1598128196 root      600        1
0x00000000 1599307845 root      600        1
0x00000000 1788903494 root      600        1
0x00000000 1787822151 root      600        1
0x00000000 1789558856 root      600        1
0x00000000 1789984841 root      600        1
0x00000000 1791131722 root      600        1
0x00000000 1791524939 root      600        1
0x00000000 2040201292 root      600        1
0x00000000 2039578701 root      600        1
0x00000000 2040627278 root      600        1
0x00000000 2041872463 root      600        1
0x00000000 2042363984 root      600        1
0x00000000 2043412561 root      600        1
0x00000000 149192786  root      600        1
0x00000000 148570195  root      600        1
0x00000000 149815380  root      600        1
0x00000000 150798421  root      600        1
0x00000000 151224406  root      600        1
0x00000000 152436823  root      600        1
0x00000000 659587160  root      600        1
0x00000000 658800729  root      600        1
0x00000000 660471898  root      600        1
0x00000000 660897883  root      600        1
0x00000000 662175836  root      600        1
0x00000000 662569053  root      600        1
0x00000000 859537502  root      600        1
0x00000000 859078751  root      600        1
0x00000000 860192864  root      600        1
0x00000000 860586081  root      600        1
0x00000000 861274210  root      600        1
0x00000000 861732963  root      600        1
0x00000000 953122916  root      600        1
0x00000000 952664165  root      600        1
0x00000000 953745510  root      600        1
0x00000000 954204263  root      600        1
0x00000000 954728552  root      600        1
0x00000000 955318377  root      600        1
0x00000000 1014890602 root      600        1
0x00000000 1109950571 root      600        1
0x00000000 1108869228 root      600        1
0x00000000 1110376557 root      600        1
0x00000000 1110933614 root      600        1
0x00000000 1111851119 root      600        1
0x00000000 1112440944 root      600        1
0x00000000 1357742193 root      600        1
0x00000000 1357119602 root      600        1
0x00000000 1358659699 root      600        1
0x00000000 1359380596 root      600        1
0x00000000 1360298101 root      600        1
0x00000000 1360920694 root      600        1
0x00000000 1610383479 root      600        1
0x00000000 1609367672 root      600        1
0x00000000 1611006073 root      600        1
0x00000000 1611989114 root      600        1
0x00000000 1612611707 root      600        1
0x00000000 1613594748 root      600        1
0x00000000 1866301565 root      600        1
0x00000000 1866039422 root      600        1
0x00000000 1866596479 root      600        1

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

0 Likes
Highlighted
New Member.

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hello Kaka

 

cd /opt/perf/bin

./glance -t

 

The following value shows if semaphore utilization has touched 100

 

--------------------------------------------------------------------------------
                              SYSTEM TABLES REPORT                  Users=    3

System Table              Available      Active     Used        Util        High
--------------------------------------------------------------------------------
Proc Table (nproc)           262144                   104                   104
File Table (nfile)            33045                     0        0.0        0.0
Shared Mem Table (shmmni)   1048576          0         24        0.0        0.0
Message Table (msgmni)      1048576         na          1        0.0        0.0
Semaphore Table (semmni)    1048576         na         25        0.0        0.0

 

 

Semaphore Table (semmni) utilization of 100 means it is reached limit

 

Thanks

Sandeep

 

 

0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi Sandeep,

 

I am on Agent 8.60.501 and there is no PA Installed. So only OS command will work for me.

 

-KAKA-

0 Likes
Highlighted
New Member.

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi Kaka

 

Please check the output of this command

 

cat /proc/sys/kernel/sem

 

If the 4th column value is 128

 

Your system has already touched the limit

 

Regards

Sandeep

Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi Sandeep,

After reboot problem is solved. I would like to what made you to point out to semaphore problem? all the subagents are running but opcmsga crashes and kernel log shows below error message.

 

Jun 12 18:31:59 servername kernel: opcmsga[20940]: segfault at 0000000000000000 rip 0000000000000000 rsp 0000000041f40028 error 14

 

A critical level kernel event has been detected from the kern log file. segfault at 0000000000000000 rip 0000000000000000 rsp 0000000041f40028This is as serious event since the priority level was marked high by the klogd daemon. A critical message from the kernel indicates the nature of the hardware problem or software inconsistency that is so severe that the system cannot continue. Corrective action should be taken immediately.



Thank You.
-KAKA-

0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hi Sandeep,

 

i am facing similar issue on on of our LINUX node. i checked for semaphore issue and see 128 in the 4th column in the output of cat /proc/sys/kernel/sem.

 

and according to your suggestion I informed the sys admin that there is semaphore issue. after checking multiple LINUX systems he confirm that on all the system he finds 128 in 4th column but there is no problem on other nodes.

 

also according to  document

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/sect-Oracle_9i_and_10g_Tuning_Guide-Setting_Semaphores-Setting_Semaphore_Parameters.html

 

128 is the max defined value but it does not confirm the uses.

 

Could you please comment on this?

 

Thank You.

-KAKA-

 

 

0 Likes
Highlighted
New Member.

Re: OA Support Tip: “Agent” components fail to start when running ovc -start

Hello Kaka

 

 

Are you getting this error on machines where sem limit has not exceeded?

 

execute "/opt/OV/lbin/eaagt/opcmsga"

 ------ Error occurred during the initialization of ITO.

 

 

Thanks

Sandeep

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.