Changed mysql password (mysql, managersetup, logger.properties) mysqld won't start

As the subject states, I followed https://community.microfocus.com/t5/HPE-ArcSight-Logger-Questions/The-password-for-the-arcsight-account-to-access-mysql-database/qaq-p/1582236 to update the arcsight user's password in mysql.

Upon restarting the server, mysqld won't boot.  If I edit my.cnf to include skip-grant-tables again and reboot the server, mysqld starts as expected.

Clearly I'm missing a location to update the new password.  Any help would be greatly appreciated.

  • Verified Answer

    Mixed from https://softwaresupport.hpe.com/group/softwaresupport/search-result/-/facetsearch/document/KM00287364?lang=en&cc=us&hpappid=202392_SSO_PRO_HPE and /cyberres/arcsight/f/arcsight-discussions/347478/the-password-for-the-arcsight-account-to-access-mysql-database-is-unknown.

     

    To change the mysql arcsight user password, follow these steps:

    • Run the command

    /etc/init.d/arcsight_services stop arcsight_web

    • Run the command 

    /etc/init.d/arcsight_services stop manager

    • Run the command 

    /etc/init.d/arcsight_services stop aps

    • Run the command 

    /etc/init.d/arcsight_services stop logger_httpd

    • Run the command 

    /etc/init.d/arcsight_services stop logger_web

    • Run the command 

    /etc/init.d/arcsight_services stop logger_servers

    • Run the command 

    /etc/init.d/arcsight_services status and make sure ONLY mysqld and postgresql services are running

    • Edit /opt/arcsight/logger/data/mysql/my.cnf
    • Add skip-grant-tables under the [mysqld] section (save changes)
    • Restart mysqld service for the changes to take effect (service arcsight_services restart mysqld)

    Note:

    mysql restart will automatically start another Arcsight services as well. Verify status (/etc/init.d/arcsight_services status)  and stop them again.

    • cd /opt/arcsight/logger/current/arcsight/bin directory
    • Issue the below command:

    ./mysql -u root

    use mysql;

    • Type flush privileges;
    • Type update user set password=PASSWORD('<NEWPASSWORD>') where user='arcsight';

    Where NEWPASSWORD is a password you have stored and kept aside for use later in this process.

    1. Run the command and press enter. This will exit mysql shell and return you back to Linux shell

    exit;

    • Backup these files from /opt/arcsight/manager/config/ (e.g. cp /opt/arcsight/manager/config/server.properties /opt/arcsight/manager/config/server.properties.old)

    server.properties, database.properties, logger.properties, logger.defaults.properties

    • Run the command and follow the instructions from such script until it's completion (use <NEWPASSWORD> when prompted).

    /opt/arcsight/manager/bin/arcsight changepassword -f /opt/arcsight/manager/config/server.properties -p dbconmanager.provider.logger.password

    • Run the command and follow the instructions from such script until it's completion (use <NEWPASSWORD> when prompted.

    /opt/arcsight/manager/bin/arcsight changepassword -f /opt/arcsight/manager/config/database.properties -p dbconmanager.provider.logger.password

    • Run the command and follow the instructions from such script until it's completion (use <NEWPASSWORD> when prompted).

    /opt/arcsight/manager/bin/arcsight changepassword -f /opt/arcsight/manager/config/database.properties -p database.user.password

    • Run the command and follow the instructions from such script until it's completion (use <NEWPASSWORD> when prompted).

    /opt/arcsight/manager/bin/arcsight changepassword -f /opt/arcsight/logger/userdata/logger/user/logger/logger.properties -p server.mysql.password

    • Run the command and follow the instructions from such script until it's completion (use <NEWPASSWORD> when prompted).

    /opt/arcsight/manager/bin/arcsight changepassword -f /opt/arcsight/logger/current/arcsight/logger/config/logger/logger.defaults.properties -p server.mysql.password

    • Edit /opt/arcsight/logger/data/mysql/my.cnf
    • Delete skip-grant-tables under the [mysqld] section (save changes)
    • Restart mysqld service for the changes to take effect (service arcsight_services restart mysqld)

    Note:

    mysql restart will automatically start another Arcsight services as well. Verify status (/etc/init.d/arcsight_services status)  and stop them again.

    • Run the command below to make sure all the services are started:

    /etc/init.d/arcsight_services start all

    • Run the command below to make sure all the services are fully started:

    /etc/init.d/arcsight_services status

  • Even I am stuck at the same point after following all the steps in the given solution.  

    Can you help what did you do to get back mysqld to run ? 

    All my services work well except mysqld and manager. After restarting mysqld removing the skip-grant-tables parameter. 

     

  • Hello,

     

    some of the steps must be run as arcsight user. 

    Before the manager service to start make sure that mysql service is started and running.

    Also to understand what's happening with the mysql and manager service take a look at mysql.log and server.std.log.

     

    Best Regards,

     

    Daniel

  • Hi Daniel,

    Thanks for your suggestion.  I was able to fix the issue few minutes later after I could see some Arcsight java processes running and mysql services even after all services where shut down. 

    I killed all those processes for a complete shutdown and restarted the mysqld after successfully. Now all services are up and functioning perfectly. 

    Thanks for all your suggestions definitely helps troubleshooting. 

    Ankit

  • Welcome,

     

    all the steps are correct but when I am dealing with MySQL stuff and the procedure requires only the mysql service to be up and running I am stoping all the service and also make sure that all of them are stopped.

    Once i am sure that all the ESM services are stopped to start each individual services i am going to /opt/arcsight/logger/current/arcsight/service/ and run the scripts of the service from there with the command:./<script> start or stop.

    Example: ./mysql start and ./mysql stop

     

    Best Regards,

     

    Daniel

     

     

  • True, I did the same. Stopped all services except mysqld during changes. 

    After that when i restarted mysqld for the last time it did not come up.  Looks like some process got  stuck. 

    But I start and stop services from /etc/init.d/arcsight_services