test4echo1 Absent Member.
Absent Member.
450 views

SmartConnectors come to a screeching halt

Jump to solution

I have many SmartConnectors installed on many RHEL5 servers.

This configuration has been running for 2+ years.

Our servers are considered security/compliance servers ... so we are required to patch them regularly.

During our maintenance period the servers were rebooted.

All SCs started up as running processes ... but no data was being processed and sent to the defined destinations.

In ESM all connectors were down.

It turns out my server team has us at RHEL 5.8 ... and support is only covered up till RHEL 5.4 (an OS released September 2009).

My SCs are out of commission - I am open to any ideas on getting up and operational.

A workaround will go a long way until I have time to reinstall the servers and the mountain of SCs.

Support refuses to assist ... saying they will be glad to help when I downgrade.

I am in support too ... it would be nice if I can tell folks to reinstall and get back to me ....

I asked for a list of specific RHEL packages to downgrade ... hoping to get back to work quickly.

They said the entire platform needed to be downgraded/reinstalled ... I know this is untrue ...

Versions after 5.4 must work - there must be a list of specific packages used by SCs.

Maybe soon we will have support for a RHEL5 release newer than 2009.

Enough about support ....

Doing some testing on my own I found you can install a new smartconnector on RHEL 5.8 ... start and stop it from the CLI ... and it works!

Unfortunately the minute you reboot the server - you are back in a failed state.  I have been unable to get the failed SC to start again.

I am not sure what gets changed yet.

If you have had this situation ... please let me know how you solved the issue.

Thanks ....

Labels (3)
0 Likes
Reply
1 Solution

Accepted Solutions
test4echo1 Absent Member.
Absent Member.

Re: SmartConnectors come to a screeching halt

Jump to solution

the issue turned out to be invalid characters in a zone name that was recently added.

It caused ALL smartconnectors to hang - with no clear indication of why it failed.

Worse yet - these characters that caused the parse issue are allowed to be input through ESM in the network model.  So you can add them ,,, it just breaks everything.


0 Likes
Reply
6 Replies
or@we-can.co.il1 Absent Member.
Absent Member.

Re: SmartConnectors come to a screeching halt

Jump to solution

Hello,

Could you reboot the server and upload 'agent.log' and 'agent.out.wrapper.log'?

Feel free to contact me at any time.

Or@we-can.co.il

+972-052-3279374

Visit us: http://www.wemeansecurity.com/

Download 'We! Analyze': https://protect724.arcsight.com/message/26360

0 Likes
Reply
test4echo1 Absent Member.
Absent Member.

Re: SmartConnectors come to a screeching halt

Jump to solution

Support looked over the logs ... here is the cause of the issue.

INFO   | jvm 1| 2012/07/19 10:00:54 | FATAL EXCEPTION:
INFO   | jvm 1| 2012/07/19 10:00:54 | com.arcsight.common.InitializationException: Exception initializing 'com.arcsight.agent.Agent': Unable to initialize agent with id [3UtZhbDgBABCGjBBRktSS9g==]
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.Agent.a(Agent.java:2621)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.Agent.K(Agent.java:448)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.Agent.a(Agent.java:999)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.Agent.d(Agent.java:827)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.Agent.main(Agent.java:1684)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at com.arcsight.agent.ph.start(ph.java:73)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at org.tanukisoftware.wrapper.WrapperManager.startInner(WrapperManager.java:1785)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at org.tanukisoftware.wrapper.WrapperManager.handleSocket(WrapperManager.java:2384)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:2696)
INFO   | jvm 1| 2012/07/19 10:00:54 |   at java.lang.Thread.run(Thread.java:662)
INFO   | jvm 1| 2012/07/19 10:00:55 | Caused by: java.lang.RuntimeException: Parser : Unknown State !(ï¿¿ with state :1).
0 Likes
Reply
catdog123 Super Contributor.
Super Contributor.

Re: SmartConnectors come to a screeching halt

Jump to solution


Is SE Linux enabled?

0 Likes
Reply
Highlighted
test4echo1 Absent Member.
Absent Member.

Re: SmartConnectors come to a screeching halt

Jump to solution

No ... SE Linux is permissive.

I would add that a reinstall allows ESM to work ...  but adding a logger destination causes the connector to fail again.  Does anyone know if logger uses OpenSSL?

0 Likes
Reply
test4echo1 Absent Member.
Absent Member.

Re: SmartConnectors come to a screeching halt

Jump to solution

the issue turned out to be invalid characters in a zone name that was recently added.

It caused ALL smartconnectors to hang - with no clear indication of why it failed.

Worse yet - these characters that caused the parse issue are allowed to be input through ESM in the network model.  So you can add them ,,, it just breaks everything.


0 Likes
Reply
Vini Acclaimed Contributor.
Acclaimed Contributor.

Re: SmartConnectors come to a screeching halt

Jump to solution

Very interesting, what characters where they?

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.