Highlighted
Regular Contributor.. Regular Contributor..
Regular Contributor..
1243 views

arcsight_web service is unavailable

ArcESM is deployed on a virtual machine on vmware esx, after migration to another cluster , ArcSight does not start.

 

aps service is unavailable
arcsight_web service is unavailable
execprocsvc service is unavailable
logger_httpd service is available
logger_servers service is unavailable
logger_web service is unavailable
manager service is unavailable
mysqld service is available
postgresql service is available

[root@tgc1-arcesm ~]# /etc/init.d/arcsight_services stop all
Monit is busy - waiting up to 2 minutes for it to free up.
Waiting for all services to start before stopping it.
Will wait up to 00:02:00.000 for all to start
FAIL: timed out awaiting start of aps service after PT00:02:00.000
FAIL: timed out awaiting start of arcsight_web service after PT00:01:59.999
arcsight_web: unavailable [
  connections to Arcsight Web Socket at localhost/127.0.0.1:9443: unavailable
  arcsight_web heartbeat "web.process": unavailable
]
FAIL: timed out awaiting start of logger_httpd service after PT00:01:59.998
FAIL: timed out awaiting start of logger_servers service after PT00:01:59.995
logger_servers: unavailable [
  logger rpc: unavailable
  arcsight_server_ready: available
]
FAIL: timed out awaiting start of logger_web service after PT00:01:59.994
FAIL: timed out awaiting start of manager service after PT00:01:59.994
manager: unavailable [
  manager rpc: unavailable
  manager heartbeat "managerup": unavailable
  manager heartbeat "server": unavailable
  manager heartbeat "server.starting": unavailable
  manager heartbeat "server.stopping": unavailable
]
FAIL: timed out awaiting start of mysqld service after PT00:01:59.990
mysqld: available [
  mysqld-jdbc: available
]
FAIL: timed out awaiting start of postgresql service after PT00:01:59.989
Waited 00:02:00.000 but all services is still starting. It cannot be stopped.
Please try again after all is running.

 

0 Likes
8 Replies
Highlighted
Respected Contributor.
Respected Contributor.

Hello, what is the error that you are getting in server.log or other services log file.? regards Sharan
0 Likes
Highlighted
Regular Contributor.. Regular Contributor..
Regular Contributor..

/opt/arcsight/logger/userdata/logger/logs/logger_server.log

there are records in the log before migration, there are no new records

 

 

/opt/arcsight/logger/userdata/logger/logs/arcsight_logger.log

Tue Apr  3 13:26:40 GMT-3 2018 [8736] /opt/arcsight/logger/current/arcsight/service/arcsight_logger start web 1
No need to update permissions/ownership
/opt/arcsight/logger/current/arcsight/service/arcsight_logger: ARCSIGHT_HOME is /opt/arcsight/logger/current/arcsight/service/../logger
/opt/arcsight/logger/current/arcsight/service/arcsight_logger: line 79: /opt/arcsight/logger/current/arcsight/service/../logger/bin/scripts/relative_paths_env.sh: No such file or directory

 

 

0 Likes
Highlighted
Respected Contributor.
Respected Contributor.

Hello,

Can you try to start the service from loggerd and the check the loggerd.log?

Did you do the migration using the snapshot? does the new machine have any IP dependency?

regards

Sharan

0 Likes
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Seems like you have different IP in the host file. Probably you need to update the host file with new IP details. 


Cheers

Gayan

Mr
0 Likes
Highlighted
Frequent Contributor.. Frequent Contributor..
Frequent Contributor..

In hosts all, ok

[root@tgc1-arcesm ~]# ifconfig
eth1      Link encap:Ethernet           inet addr:172.20.17.217

 cat /etc/hosts

172.20.17.217 tgc1-arcesm.tgc1.local
172.20.17.218 tgc1-ArcWCon.tgc1.local
172.20.17.219 tgc1-ArcLCon.tgc1.local

 

0 Likes
Highlighted
Frequent Contributor.. Frequent Contributor..
Frequent Contributor..

copy /opt/arcsight/logger/BL1458/arcsight/logger/bin/scripts/relative_paths_env.sh
to /opt/arcsight/logger/current/arcsight/logger/bin/scripts/relative_paths_env.sh

aps service is unavailable
arcsight_web service is unavailable
execprocsvc service is available
logger_httpd service is available
logger_servers service is available
logger_web service is available
manager service is initializing
mysqld service is available
postgresql service is available

 

 

 

0 Likes
Highlighted
Respected Contributor.
Respected Contributor.

Looks like a permission issue.

 

Can you verify the permissions of the /current folder and the subfolders before and after.?

 

regards

Sharan

0 Likes
Highlighted
Frequent Contributor.. Frequent Contributor..
Frequent Contributor..

SQL base broken???

 

 

[2018-04-04 11:10:01,635][ERROR][default.com.arcsight.common.persist.mysql.MysqlDatabaseInfoBroker] Unable to get columns for table 'arc_trend_runs': Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
[2018-04-04 11:10:01,638][ERROR][default.com.arcsight.common.persist.mysql.MysqlDatabaseInfoBroker]
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3715)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3604)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4155)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781)
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1569)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2718)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2605)
    at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
    at com.mysql.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2604)
    at com.arcsight.common.persist.DatabaseInfoBrokerBase.getColumnNamesForTable(DatabaseInfoBrokerBase.java:497)
    at com.arcsight.server.verification.SchemaValidator.validateColumnsExist(SchemaValidator.java:472)
    at com.arcsight.server.verification.SchemaValidator.validateSchemaForOtherTables(SchemaValidator.java:134)
    at com.arcsight.server.verification.SchemaValidator.validateSchema(SchemaValidator.java:120)
    at com.arcsight.server.NGServer.runSchemaValidation(NGServer.java:1881)
    at com.arcsight.server.NGServer.initialize(NGServer.java:873)
    at com.arcsight.server.NGServer.main(NGServer.java:4531)
    at com.arcsight.server.plugins.manager.Service.start(Service.java:36)
    at com.arcsight.dm.plugins.core.Application.initializeServices(Application.java:141)
    at com.arcsight.dm.plugins.core.Application.run(Application.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.arcsight.server.WrapperLauncher.start(WrapperLauncher.java:102)
    at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2965)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3161)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3615)
    ... 45 more
[2018-04-04 11:10:01,640][ERROR][default.com.arcsight.server.NGServer]
com.arcsight.common.persist.PersistenceException: Unable to get columns for table 'arc_trend_runs': Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
    at com.arcsight.common.persist.DatabaseInfoBrokerBase.getColumnNamesForTable(DatabaseInfoBrokerBase.java:531)
    at com.arcsight.server.verification.SchemaValidator.validateColumnsExist(SchemaValidator.java:472)
    at com.arcsight.server.verification.SchemaValidator.validateSchemaForOtherTables(SchemaValidator.java:134)
    at com.arcsight.server.verification.SchemaValidator.validateSchema(SchemaValidator.java:120)
    at com.arcsight.server.NGServer.runSchemaValidation(NGServer.java:1881)
    at com.arcsight.server.NGServer.initialize(NGServer.java:873)
    at com.arcsight.server.NGServer.main(NGServer.java:4531)
    at com.arcsight.server.plugins.manager.Service.start(Service.java:36)
    at com.arcsight.dm.plugins.core.Application.initializeServices(Application.java:141)
    at com.arcsight.dm.plugins.core.Application.run(Application.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.arcsight.server.WrapperLauncher.start(WrapperLauncher.java:102)
    at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2965)
[2018-04-04 11:10:01,647][INFO ][default.com.arcsight.server.ab] Shutdown Started.
[2018-04-04 11:10:01,649][INFO ][default.com.arcsight.agent.wf] Shutting Down Agent Framework Version [6.8.0.1896.0]
[2018-04-04 11:10:01,676][INFO ][default.com.arcsight.server.StatusInfo] Configuration [PrintMemStdOut:true PrintMemLog:true MemoryMultiplicity:1 PrintServiceListStdOut:false PrintServiceListLog:true ServiceListMultiplicity:30]
[2018-04-04 11:10:01,677][INFO ][default.com.arcsight.server.management.ManagementAgent] Registered MBean 'Arcsight:service=Status'.
[2018-04-04 11:10:01,766][INFO ][default.com.arcsight.server.management.ManagementAgent] Registered MBean 'Arcsight:service=ZoneEngineManager'.
[2018-04-04 11:10:01,809][ERROR][default.com.arcsight.common.persist.DatabaseResourceManager]
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.Util.getInstance(Util.java:384)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:973)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
    at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1321)
    at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1313)
    at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5311)
    at com.bitmechanic.sql.PooledConnection.setAutoCommit(PooledConnection.java:424)
    at com.arcsight.server.database.JDBCPoolConnectionProvider.getConnection(JDBCPoolConnectionProvider.java:154)
    at com.arcsight.server.database.DatabaseConnectionManager.getConnection(DatabaseConnectionManager.java:116)
    at com.arcsight.common.persist.DatabaseResourceManager.getConnection(DatabaseResourceManager.java:427)
    at com.arcsight.common.persist.DatabaseResourceManager.getConnection(DatabaseResourceManager.java:363)
    at com.arcsight.server.search.lucene.IndexDatastoreDirectory.getDBUrlHash(IndexDatastoreDirectory.java:324)
    at com.arcsight.server.search.lucene.IndexDatastoreDirectory.getDirectory(IndexDatastoreDirectory.java:184)
    at com.arcsight.server.search.SearchService._init(SearchService.java:144)
    at com.arcsight.server.search.SearchService.<init>(SearchService.java:112)
    at com.arcsight.server.search.SearchService.getInstance(SearchService.java:129)
    at com.arcsight.server.l.run(l.java:447)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3715)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3604)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4155)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781)
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1569)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2718)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2605)
    at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
    at com.mysql.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2604)
    at com.arcsight.common.persist.DatabaseInfoBrokerBase.getColumnNamesForTable(DatabaseInfoBrokerBase.java:497)
    at com.arcsight.server.verification.SchemaValidator.validateColumnsExist(SchemaValidator.java:472)
    at com.arcsight.server.verification.SchemaValidator.validateSchemaForOtherTables(SchemaValidator.java:134)
    at com.arcsight.server.verification.SchemaValidator.validateSchema(SchemaValidator.java:120)
    at com.arcsight.server.NGServer.runSchemaValidation(NGServer.java:1881)
    at com.arcsight.server.NGServer.initialize(NGServer.java:873)
    at com.arcsight.server.NGServer.main(NGServer.java:4531)
    at com.arcsight.server.plugins.manager.Service.start(Service.java:36)
    at com.arcsight.dm.plugins.core.Application.initializeServices(Application.java:141)
    at com.arcsight.dm.plugins.core.Application.run(Application.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.arcsight.server.WrapperLauncher.start(WrapperLauncher.java:102)
    at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2965)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3161)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3615)
    ... 45 more
[2018-04-04 11:10:01,812][ERROR][default.com.arcsight.server.search.lucene.IndexDatastoreDirectory]
com.arcsight.common.persist.PersistenceException: Unable to get connection: No operations allowed after connection closed.
    at com.arcsight.common.persist.DatabaseResourceManager.getConnection(DatabaseResourceManager.java:470)
    at com.arcsight.common.persist.DatabaseResourceManager.getConnection(DatabaseResourceManager.java:363)
    at com.arcsight.server.search.lucene.IndexDatastoreDirectory.getDBUrlHash(IndexDatastoreDirectory.java:324)
    at com.arcsight.server.search.lucene.IndexDatastoreDirectory.getDirectory(IndexDatastoreDirectory.java:184)
    at com.arcsight.server.search.SearchService._init(SearchService.java:144)
    at com.arcsight.server.search.SearchService.<init>(SearchService.java:112)
    at com.arcsight.server.search.SearchService.getInstance(SearchService.java:129)
    at com.arcsight.server.l.run(l.java:447)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.Util.getInstance(Util.java:384)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:973)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
    at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1321)
    at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1313)
    at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5311)
    at com.bitmechanic.sql.PooledConnection.setAutoCommit(PooledConnection.java:424)
    at com.arcsight.server.database.JDBCPoolConnectionProvider.getConnection(JDBCPoolConnectionProvider.java:154)
    at com.arcsight.server.database.DatabaseConnectionManager.getConnection(DatabaseConnectionManager.java:116)
    at com.arcsight.common.persist.DatabaseResourceManager.getConnection(DatabaseResourceManager.java:427)
    ... 8 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3715)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3604)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4155)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781)
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1569)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2718)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2605)
    at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
    at com.mysql.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2604)
    at com.arcsight.common.persist.DatabaseInfoBrokerBase.getColumnNamesForTable(DatabaseInfoBrokerBase.java:497)
    at com.arcsight.server.verification.SchemaValidator.validateColumnsExist(SchemaValidator.java:472)
    at com.arcsight.server.verification.SchemaValidator.validateSchemaForOtherTables(SchemaValidator.java:134)
    at com.arcsight.server.verification.SchemaValidator.validateSchema(SchemaValidator.java:120)
    at com.arcsight.server.NGServer.runSchemaValidation(NGServer.java:1881)
    at com.arcsight.server.NGServer.initialize(NGServer.java:873)
    at com.arcsight.server.NGServer.main(NGServer.java:4531)
    at com.arcsight.server.plugins.manager.Service.start(Service.java:36)
    at com.arcsight.dm.plugins.core.Application.initializeServices(Application.java:141)
    at com.arcsight.dm.plugins.core.Application.run(Application.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.arcsight.server.WrapperLauncher.start(WrapperLauncher.java:102)
    at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2965)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3161)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3615)
    ... 45 more
[2018-04-04 11:10:01,826][INFO ][default.com.arcsight.server.management.ManagementAgent] Registered MBean 'Arcsight:service=SearchService'.
[2018-04-04 11:10:01,834][INFO ][default.com.arcsight.server.management.ManagementAgent] Registered MBean 'Arcsight:service=IndexResources'.
[2018-04-04 11:10:01,835][INFO ][default.com.arcsight.server.l] Completed.
[2018-04-04 11:10:01,837][INFO ][default.com.arcsight.util.host.HostSystemInfoProviderImplLinux$StatCollectorThread] Stopped explicitly.
[2018-04-04 11:10:01,844][ERROR][default.com.arcsight.server.database.JDBCPoolConnectionProvider$ConnectionCloseListener]
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.Util.getInstance(Util.java:384)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:973)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
    at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1321)
    at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1313)
    at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:5046)
    at com.bitmechanic.sql.PooledConnection.rollback(PooledConnection.java:436)
    at com.arcsight.server.database.JDBCPoolConnectionProvider$ConnectionCloseListener.beforeConnectionClose(JDBCPoolConnectionProvider.java:479)
    at com.bitmechanic.sql.ConnectionPool.closeConnection(ConnectionPool.java:371)
    at com.bitmechanic.sql.ConnectionPool.closeConnections(ConnectionPool.java:345)
    at com.bitmechanic.sql.ConnectionPool.reapIdleConnections(ConnectionPool.java:288)
    at com.arcsight.server.database.JDBCPoolConnectionProvider.dispose(JDBCPoolConnectionProvider.java:199)
    at com.arcsight.server.database.DatabaseConnectionManager.dispose(DatabaseConnectionManager.java:177)
    at com.arcsight.server.m.run(m.java:275)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 88 milliseconds ago.  The last packet sent successfully to the server was 88 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3715)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3604)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4155)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781)
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1569)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2718)
    at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2605)
    at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
    at com.mysql.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2604)
    at com.arcsight.common.persist.DatabaseInfoBrokerBase.getColumnNamesForTable(DatabaseInfoBrokerBase.java:497)
    at com.arcsight.server.verification.SchemaValidator.validateColumnsExist(SchemaValidator.java:472)
    at com.arcsight.server.verification.SchemaValidator.validateSchemaForOtherTables(SchemaValidator.java:134)
    at com.arcsight.server.verification.SchemaValidator.validateSchema(SchemaValidator.java:120)
    at com.arcsight.server.NGServer.runSchemaValidation(NGServer.java:1881)
    at com.arcsight.server.NGServer.initialize(NGServer.java:873)
    at com.arcsight.server.NGServer.main(NGServer.java:4531)
    at com.arcsight.server.plugins.manager.Service.start(Service.java:36)
    at com.arcsight.dm.plugins.core.Application.initializeServices(Application.java:141)
    at com.arcsight.dm.plugins.core.Application.run(Application.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.arcsight.server.WrapperLauncher.start(WrapperLauncher.java:102)
    at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2965)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3161)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3615)
    ... 45 more
[2018-04-04 11:10:01,850][INFO ][default.com.arcsight.common.flow.FlowManager] Flow manager initialized.
[2018-04-04 11:10:01,856][INFO ][default.com.arcsight.agent.lb.b] Closing cache [SecurityEventBroker].
[2018-04-04 11:10:01,857][INFO ][default.com.arcsight.agent.lb.b] Cache [SecurityEventBroker] closed.
[2018-04-04 11:10:01,859][INFO ][default.com.arcsight.server.ab] Shutdown Completed.
<CODE MAP: '6.8.0.1896.0'>
<ArcSight Version: 6.8.0.1896.0>
[2018-04-04 11:10:36,117][INFO ][default.com.arcsight.server.NGServer] Using log implementation 'com.arcsight.common.log.FileLog4JLog@4387384'.
[2018-04-04 11:10:36,117][INFO ][default.com.arcsight.server.NGServer] --------------------------
[2018-04-04 11:10:36,117][INFO ][default.com.arcsight.server.NGServer] ArcSight Manager start up.
[2018-04-04 11:10:36,117][INFO ][default.com.arcsight.server.NGServer] --------------------------
[2018-04-04 11:10:36,118][INFO ][default.com.arcsight.server.management.ManagementAgent] Using MBean server 'com.sun.jmx.mbeanserver.JmxMBeanServer@4c7d5d85'.
[2018-04-04 11:10:36,118][INFO ][default.com.arcsight.server.management.ManagementAgent] Registered MBean 'Arcsight:service=LogManager'.
[2018-04-04 11:10:36,283][INFO ][default.com.arcsight.common.resourcebundle.ResourceBundle] No property rsbundle.missed.prop.timeout.sec found. Will use default value
[2018-04-04 11:10:36,283][INFO ][default.com.arcsight.common.resourcebundle.ResourceBundle] Config: expiration timeout=-1 msec.
[2018-04-04 11:10:36,394][INFO ][default.com.arcsight.server.NGServer] License initialized. (2 s)
[2018-04-04 11:10:36,601][INFO ][default.com.arcsight.crypto.SSLConfiguration] SECURITY PROVIDERS INITIALIZED:
  PROVIDER[0]: SUN (sun.security.provider.Sun)
  PROVIDER[1]: SunRsaSign (sun.security.rsa.SunRsaSign)
  PROVIDER[2]: SunEC (sun.security.ec.SunEC)
  PROVIDER[3]: SunJSSE (com.sun.net.ssl.internal.ssl.Provider)
  PROVIDER[4]: SunJCE (com.sun.crypto.provider.SunJCE)
  PROVIDER[5]: SunJGSS (sun.security.jgss.SunProvider)
  PROVIDER[6]: SunSASL (com.sun.security.sasl.Provider)
  PROVIDER[7]: XMLDSig (org.jcp.xml.dsig.internal.dom.XMLDSigRI)
  PROVIDER[8]: SunPCSC (sun.security.smartcardio.SunPCSC)
  PROVIDER[9]: BC (org.bouncycastle.jce.provider.BouncyCastleProvider)
  PROVIDER[10]: NDBS (edu.cmu.sei.cbs.ndbs.NDBS)


0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.