Commander
Commander
2674 views

ArcSight manager service not starting and web service is in mixed_statuses

ArcSight manager service not starting and web service is in mixed_statuses.

The Manager Service is "Unavailable" and sometimes "initializing" and the Web Service is in "mixed_statuses".

We have tried restarting the problematic services. But since the services are in "mixed_statuses" and "initializing" so the services are not getting started neither getting getting stopped.

All the services were running properly previously hence there is no issue with port.

6046 [ERROR] /opt/arcsight/logger/current/local/mysql/libexec/mysqld: Incorrect key file for table './mysql/db'; try to repair it

134 [ERROR] Fatal error: Can't open and lock privilege tables: Incorrect key file for table './mysql/db'; try to repair it

1 0x7f71be098020-1]flag=18, query: SELECT event_id, external_id, name, message, event_type, start_time, end_time, manager_receipt_time, app_protocol,

trans_protocol, vulnerability, raw_event, bytes_in, bytes_out, session_id, generator, crypto_signature, concentrator_agents, concentrator_devices,

It seems table "DB" in MySQL is corrupted.

while repairing we are getting the error :  Can't connect to local MySQL server through socket '/opt/arcsight/logger/current/runtime/mysql.sock' (2).

Kindly help.

Labels (2)
Tags (1)
0 Likes
9 Replies
Absent Member.
Absent Member.

Hello ​,

Problems like this, especially involving system downtime/outages, are best addressed by contacting HPE ArcSight Support and working through a support ticket.  At first glance, this does appear to be some sort of database corruption, which results in a domino effect on the services - mysqld down means manager will be initializing/unavailable, which in turn means web will always be in "mixed_status" because it depends on manager.

Thanks and Regards,

Michael Lumsden

ESM Product Manager

0 Likes
Commander
Commander

Thanks Michael,

We have lodge a call to support team and its resolved at the moment.

0 Likes
Cadet 1st Class
Cadet 1st Class

Hi Sagar, how did this issue got resolved. I meant what changes the support team did to make it works again.

0 Likes
Commander
Commander

Our database was corrected which couldn't be recovered. database was restored from backup.

The main issue was stopping the services. They had to forcefully close all the services including monit service so that services doesn't start automatically.

0 Likes
Commander
Commander

Our database was corrupted which couldn't be recovered. database was restored from backup.

The main issue was stopping the services. They had to forcefully close all the services including monit service so that services doesn't start automatically.

0 Likes
Commander Commander
Commander

Hi Sagar,

I had the same problem, could you explain step by step to solve this?

I really apreciate your help!

Regrads,

Achmad Abrar

0 Likes
Commander
Commander

Hi Achmad,

Find the moint service.

ps -ef |grep monit

Kill the monit services forcefully and monitor if the service is getting up once again.

watch -d ps -ef | grep monit

kill -9 <Process ID>

OR

You can also do

Kill /bin/bash /etc/init.d/arcsight_services start monit

OR

Run /etc/init.d/arcsight_service stop monit

We did tried below two commands to stop the monit services but it didn't help.

After monit service is off stop services one by one and start one by one.

Sequence of starting the service :

/sbin/service arcsight_services start mysqld

/sbin/service arcsight_services start logger

/sbin/service arcsight_services start manager

/sbin/service arcsight_services start arcsight_web

0 Likes
Commander Commander
Commander

Hi Sagar,

Thankyou!

Regards,

Achmad Abrar

0 Likes
Commander
Commander

You are most welcome

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.