Anyone else having issues with 188.8.131.5296 ?
Just noticed on two different conapps shortly after I applied 184.108.40.20696 the EPS out on the containers dropped to 0. EPS in was normal but nothing out. Logs on the container show Connection to [loggername] port 443 failed ping test and then a bunch of java exceptions. I rebooted the entire conapp, no change. Smart connectors seem find though, just the conapps.
Also seeing [2015-04-22 14:30:07,145][FATAL][default.com.arcsight.agent.loadable.transport.event._AgentCEFFileEventTransport][createNewFile] File [/opt/arcsight/connector_1/current/user/agent/cef/2015-04-22-14-30-07.cef] could not be created! because there is no cef directory on the conapps.
Shortly after I posted this all of my smart connectors started doing the same thing. It may have something to do with the fact that all of our connectors go to Loggers instead of ESM, dunno, either way we are having to do an emergency rollback since our EPS out on all of our connectors is literally 0.
we have the same issue, all connectors was upgraded to 7.1.2 and no Events was send to the Logger, but all Events was send to the ESM Destination. After I make a Rollback to 7.1.1 all Events from the cache was send to the logger. I see in the Log File a error from the WatchDog Thread. It's looks like the connection to the logger will be test with a ping to 443 and this don't work.
Well I'm glad it wasnt just me, I was told that nobody else had reported any issues. The rollback immediately fixed all of my connectors except for 2 of my 6 checkpoint connectors running on conapps that would not downgrade. I had to manually rename the folders via ssh to get those connectors downgraded and running again.
I guess QA didnt test the new build with connectors registered to logger, just ESM. I have several bugs created with support about issues\errors on the ESM for the loggers that are sending events to the ESM. My guess is that the developers are assuming all customers are sending events from the Connectors to the ESM, not from the Connectors to logger, which is why they arent seeing these bugs we are having. We would have a lot less issues with our ESM if we re-designed it so that the connectors send logs directly to the ESM but we cant because we have to have 2 active DR feeds and 2 standby HA feeds for compliance reasons.
Thanks for the heads up, I think this will save a lot of headaches as we were about to test deployment of the new connector.
This is obviously a serious issue that should have been caught in QA - its not exactly subtle, and its effect is about as severe as you can get.
Is this issue only in case of ConApp or it persists with the software based connector deployments as well. Has anyone tested, if so please update your observations.
Thanks in advance.
I have yesterday a Support Session with HP and i upload the log.
We test it with ConApp and software based connector, every time we upgraded a existing connector to 7.1.2 the problem appeared, if we make a fresh installation not.
Once this issue occurs, have you tried re-registering the destination OR removing & adding the current Logger destination which I guess it should work.
I do wonder if it would not be better for HP to just set this up in their lab and reproduce it themselves instead of having customers create tickets, reproduce, upload logs, test workarounds etc.... I would think it would be much faster and more effective if HP Dev\QA\Support does the investigation\testing rather than relying on the customer to do it for them. I could understand having the customer reproduce and investigate when the issues are complicated and specific to a customers environment, but this seems like a very simple and easily reproducible scenario that is not customer specific.
Do you agree?
I completely agree with you. HP should be testing various scenarios and stability of the connectors in their lab before releasing the connector upgrades. And also additionally I also think that HP should be starting the concept of Beta and Stable version release which will drastically reduce major issues and also HP would be less dependent on client to detect the issues and report it....
Hope it makes sense.