(BSM) Support Tip : BSM 9.2x on Linux - Login not possible - only GW restart helps
Releasing the information here after multiple customers faced the issue,
although it's all documented in the manual set.
Environment : BSM 9.26 IP1 on Linux, HA setup with 2 x GW, 2 x DPS, 1 Loadbalancer + LDAP integration for user authentication
Unable to login to the BSM GUI once in a while.
This is the error message returned:
This may be due to incorrect login name/password combination. Enter
your login name and password again. If you see this error message
again, please notify your HP Business Service Management
Once in a while this message is reported in the browser:
JBWEB000065: HTTP Status 500 - bsm.customer.com
JBWEB000309: type JBWEB000066: Exception report
JBWEB000068: message bsm.customer.com
JBWEB000069: description JBWEB000145: The server encountered an internal error that prevented it from fulfilling this request.
JBWEB000072: note JBWEB000073: The full stack trace of the root cause in the JBoss Web/7.4.2.Final logs
Logging in with the admin account doesn't work either.
This problem was present before, the solution was to restart the BSM on both of GWs and after that everything was working.
But this works only for few days and then the problem appears again.
When the issue shows up, JBoss also shows a very high CPU utilization on both GW servers.
jboss7_boot.log on GW is full with warnings of this kind:
2016-04-28 09:18:37,367 - [0m[33m09:18:37,367 WARNING [sun.rmi.transport.tcp] (RMI TCP Accept-0)
RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,localport=45447] throws: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
explains it all.
As per the BSM 9.26 documentation
BSM System Requirements and Support Matrixes, Chapter 2: BSM System Requirements.
Linux Resource Limits Resource limits (open files and max user processes) on Linux machines
must be set to 30000 or higher.
Here ulimit was set to 8000 only,
increasing the ulimit –n to 30000 resolved the problem.
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.