Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
Codefire1 Absent Member.
Absent Member.
4980 views

ArcSight Logger Process Fails To Start

Jump to solution

I bounced my Logger the other day (no changes made to system); when it came back up (and it appears to have done so) all the accounts were locked out of the web interface. Logging into the CLI shows that everything seems to be running as expected except one of the processes failed to start.

/opt/local/monit/bin/monit summary

The Monit daemon 5.1.1 uptime: 7h 6m

Process 'postgresql'                running

File 'arcsight_primary_storage_available' accessible

Process 'mysqld'                    running

Process 'servers'                   running

Process 'receivers'                 running

Process 'processors'                running

Process 'web'                       running

Process 'reportengine'              running

File 'arcsight_server_ready'        accessible

Process 'insp'                      Execution failed

Process 'apache'                    running

Process 'connector'                 running

Process 'aps'                       running

System 'MyArcsightServer'  running

I have tried to restart the "insp" process with:

/opt/local/monit/bin/monit stop insp

/opt/local/monit/bin/monit start insp

And it shows the expected messages; process not monitored - stop pending, process not monitored, process not monitored - start pending, and process initializing. Followed, unfortunately, by process execution failed.

I looked through all the logs on the system and found nothing even remotely mentioning "insp" or anything about a process failing to load.


Anyone come across this before or know how I could proceed?


Thanks in advance

Labels (2)
0 Likes
1 Solution

Accepted Solutions
Highlighted
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Re-install of perl fixed the issue.

0 Likes
14 Replies
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Tried restarting the OS and rebooted the server as well.

0 Likes
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Found this in: /opt/local/monit/log/monit.log:

[EST Dec 10 06:50:01] info : Starting monit daemon with http interface at [*:2812]
[EST Dec 10 06:50:01] info : Starting monit HTTP server at [*:2812]
[EST Dec 10 06:50:01] info : monit HTTP server started
[EST Dec 10 06:50:01] info : 'MyArcsightServer' Monit started
[EST Dec 10 06:50:01] error: 'insp' process is not running
[EST Dec 10 06:50:01] info : 'insp' trying to restart
[EST Dec 10 06:50:01] info : 'insp' start: /opt/local/monit/watchdog/insp.run
[EST Dec 10 06:50:31] error: 'insp' failed to start
[EST Dec 10 06:51:02] error: 'insp' process is not running
[EST Dec 10 06:51:02] info : 'insp' trying to restart
[EST Dec 10 06:51:02] info : 'insp' start: /opt/local/monit/watchdog/insp.run
[EST Dec 10 06:51:32] error: 'insp' failed to start
[EST Dec 10 06:52:02] error: 'insp' process is not running
[EST Dec 10 06:52:02] info : 'insp' trying to restart

When I go to; /opt/local/monit/watchdog/insp.run; it's a shell script that pretty much runs: /opt/ENIRA/scripts/insp.cgi

0 Likes
Highlighted
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Re-install of perl fixed the issue.

0 Likes
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Issue occurred on another of our Loggers and the perl package reinstall fixed it right away.

0 Likes
hemant1989 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Hi,

I have installed a logger and i am facing same kind of problem, for me

Process 'mysqld'                    Execution Failed

Process 'reportengine'             Execution failed

rest other process are in running state.

I dnt knw if installing perl package will help, but still want to give a try please let me know where i will find perl package.

0 Likes
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

There should be a copy in:

/opt/updates/logger-6288/RPMS/arcsight-perl-5.8.8-18.x86_64.rpm

(Your exact dir structure may be slightly different.)

But, the copy I used was from:

/var/cache/yum/ape/packages/arcsight-perl-5.8.8-17.x86_64.rpm

(Your exact dir structure may be slightly different.)

0 Likes
hemant1989 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

could you also provide me the steps for re-installing the perl.

0 Likes
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Drawing on memory so please verify before implementing.

*Shut down all ArcSight processes*

# /opt/local/monit/bin/monit stop all

*Verify that they're stopped*

# /opt/local/monit/bin/monit summary

*Find current install of Perl*

# rpm -qa | grep perl

arcsight-mod_perl-2.0.2-5

arcsight-perl-5.8.8-27

perl-5.8.8-38.el5

*Remove arcsight perl installations*

# rpm -ev arcsight-mod_perl-2.0.2-5

# rpm -ev arcsight-perl-5.8.8-27

*Install backup copies*

# rpm -Uvh  /var/cache/yum/ape/packages/arcsight-perl-5.8.8-17.x86_64.rpm

# rpm -Uvh /var/cache/yum/ape/packages/arcsight-mod_perl-2.0.2-4.x86_64.rpm

*Start ArcSight processes*

# /opt/local/monit/bin/monit start all

*Verify*

# /opt/local/monit/bin/monit summary

If you're still having an issue there was a way to test the arcsight perl install. But I don't recall what it was. I think it had to do with attempting to run one of the arcsight perl modules and tracking the error message.

0 Likes
Codefire1 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Meanwhile, in re-reading your issue; have you looked at the log files? I'd wager an issue with MySQL is not perl related. Check the ArcSight and MySQL logs before messing with anything.

0 Likes
hemant1989 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

where will i find the log file?

0 Likes
hemant1989 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

As checked in the logs, path:

\opt\arcsight\userdata\logger\logs\logger_mysqld

found below logs. It seems some DB tables are missing.

http://welcome.hp-ww.com/img/s.gif 130912 2:48:13 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
     130912 2:48:13 [ERROR] /opt/local/mysql/libexec/mysqld: Incorrect information in file: './mysql/host.frm'
http://welcome.hp-ww.com/img/s.gif 130912 2:48:13 [ERROR] Fatal error: Can't open and lock privilege tables: Incorrect information in file: './mysql/host.frm'


Restoring to factory default setting helped me. All services are running fine now.


Also learnt that Logger model L7500 has this bug and and if anyone faces above problem in fresh installation should go for factory reset.

0 Likes
pravin2all Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

We face same issue for logger, I have checked logger external storage is full.

I have delete old file from path /opt/data/logger and then i restart monit.

Its works for me

[root@pravinlogger1 logger]# df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda3              99G   33G   61G  36% /

/dev/sda2              99M   31M   64M  33% /boot

none                  3.9G     0  3.9G   0% /dev/shm

/dev/sdb1             3.6T  3.6T   22m 100% /opt/data

[root@pravinlogger1 logger]#


0 Likes
vaish_11 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Hello Hemant,

We have recently installed L7500 Logger and are facing the same issue. The ArcSight Logger Admin Guide v5.3 states the below instruction, however, I don't see any such message when I try to reboot. Could you please tell me how you performed the factory reset? :

Once the following message appears on the screen, press any key on your keyboard.

This message is displayed for a very short time, therefore, make sure you press a key

on your keyboard quickly; otherwise, the Logger will continue to boot normally.

Press any key to enter the menu

Booting Red Hat Enterprise Linux Server in 2 seconds...

0 Likes
hemant1989 Absent Member.
Absent Member.

Re: ArcSight Logger Process Fails To Start

Jump to solution

Hi Vaishnavi,

When you get below message at the time of reboot

"Press any key to enter the menu

Booting Red Hat Enterprise Linux Server in 2 seconds..."

press any key, you will enter the setup. First thing you have to do is select the drive and then select to verify it. Once you verify it again select the drive and restore it.

Thats it. Once done. just reboot it and you are good to go. 🙂

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.