Highlighted
Contributor.
Contributor.
749 views

Load Balancing on ArcSight Logger 6.0

Jump to solution

Good morning Protect724 Community!  I apologize in advance if this is something that has been previously discussed, but I did spend some time searching and could not find information that would apply to my situation.  We have just purchased two HP ArcSight 6.0 Logger appliances, and I have configured them for our network, set up the Peer Connection between them, and I am currently forwarding information from our existing SIEM solution (which is slated to be decommissioned) to begin populating data into the loggers for searching and testing.

Am I correct in understanding that the only way to truly load balance the loggers (without purchasing a load balancer device) is from the SmartConnector or source side?  In the case of capturing events from our Firewall, since currently it is responsible for the bulk of the events recording thus far (almost 10x more events than everything else combined), how to I spread those events between our two Logger appliances?

Thank you in advance for any and all assistance!

Labels (2)
0 Likes
1 Solution

Accepted Solutions
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Dear Andre,

According last information I have, there is no real Load balancing solution.

You have to configure by your self the source of the smartconnector to have data approximatively load balanced between your lwo loggers to be sure that the peering feature will be the most efficient.

There is a failover feature on Logger but it is just to have an high availablity solution in case of one Logger is anavailable.

In you case, I choose Logger 2 as failover of Logger 1 for connector 1 et vice-versa for connector 2.

But for the load balancing, you have to study make statistics to share sources between your two loggers.

Like


FW1,FW3,FW5 => Connector 1 => Logger1 (Primary) Logger2 (failover)

FW2,FW4,FW6 => Connector 2 => Logger2 (Primary) Logger1 (failover)

If nbr of events of this two groups is more or less equal on a long term periode as 2days, 1week, etc...

or you adapt the choice.

If you find another solution, I am completely interested because we have to design a new logger architecture based on Logger v6.0 with peering feature highly efficient.

I made a lots of tests on POC, if data is not properly load balanced through your logger it is not so efficient which is normal because each logger make the query on his own logs and the slower logger will be your result + short time of reassembly of logs to one logger.

You may use a load-balancer in front of smartconnectors but I do not think it is working for all smartconnector types like API

Syslog should work but no tested.

I hope this information will help you.

If I receive more useful information from HP ArcSight PS, I will come to you.

Thanks

Regards

Michael

View solution in original post

0 Likes
7 Replies
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Dear Andre,

According last information I have, there is no real Load balancing solution.

You have to configure by your self the source of the smartconnector to have data approximatively load balanced between your lwo loggers to be sure that the peering feature will be the most efficient.

There is a failover feature on Logger but it is just to have an high availablity solution in case of one Logger is anavailable.

In you case, I choose Logger 2 as failover of Logger 1 for connector 1 et vice-versa for connector 2.

But for the load balancing, you have to study make statistics to share sources between your two loggers.

Like


FW1,FW3,FW5 => Connector 1 => Logger1 (Primary) Logger2 (failover)

FW2,FW4,FW6 => Connector 2 => Logger2 (Primary) Logger1 (failover)

If nbr of events of this two groups is more or less equal on a long term periode as 2days, 1week, etc...

or you adapt the choice.

If you find another solution, I am completely interested because we have to design a new logger architecture based on Logger v6.0 with peering feature highly efficient.

I made a lots of tests on POC, if data is not properly load balanced through your logger it is not so efficient which is normal because each logger make the query on his own logs and the slower logger will be your result + short time of reassembly of logs to one logger.

You may use a load-balancer in front of smartconnectors but I do not think it is working for all smartconnector types like API

Syslog should work but no tested.

I hope this information will help you.

If I receive more useful information from HP ArcSight PS, I will come to you.

Thanks

Regards

Michael

View solution in original post

0 Likes
Highlighted
Contributor.
Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Michael,

Thank you for your reply.  This information helps me to decide on an architecture that will best deploy logger in our environment.  I will take an in depth look at syslog because it is the firewall events that I want to distribute.  Please keep me posted if you hear any more information, I am relatively new into this project, and I am looking to learn as much as I can as quickly as I can.

0 Likes
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

A feature to enable connectors to send to multiple loggers is currently being worked on. Unfortunately I cannot discuss on Protect724 time lines. If important to your project, please contact your HP account representative to arrange a roadmap discussion.

~ Ofer

ArcSight product management

0 Likes
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Dear Ofer Shezaf,

Great news. I am completely interested. I will contact my HP account.

This feature has a specific name? When this feature would be available?

Thanks

Regards

Michael

0 Likes
Highlighted
Contributor.
Contributor.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Ofer,

I too, echo Michael's interest in this feature, and would like more information if it is available.  Thank you for informing us of this, and I look forward to speaking with my HP representative.

0 Likes
Highlighted
Absent Member.
Absent Member.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

I'm definitely interested in this as well!

Andrea,

We are doing our own sort of Load Balancing that may interest you. We have our syslog logs sent to a server. From there we have syntax in the syslog.conf file that parses the incoming logs to split them in to a new file for the firewall logs and and for the rest of the syslog logs. We have a normal syslog Smartconnector that reads the new firewall file and another one that reads the everything-else-syslog file. From there you can have each connector send to a separate Logger.

If/when ArcSight brings this type of functionality to a connector it would be awesome for simple load balancing.

0 Likes
Highlighted
Absent Member.
Absent Member.

Re: Load Balancing on ArcSight Logger 6.0

Jump to solution

Below is an example of a failover I did. The primary software loggers had dual destination out one to ESM the other to failover Hardware Loggers. The failover from the hardware to the loggers had a filter that looked for events only coming from fail over receiver. This setup gave us a couple advantages

  1. Data was duplicated between two sites
  2. Analysts only had to point consoles at secondary ESM during outage
  3. Network outage between connectors and primary site was dynamically detected and event flow shifted.
  4. Both ESM and Loggers were made whole again when outage was resolved.

Not perfect, and it wouldn't do well for extended multiple day outages due the high level of caching that would occur between hardware and software. But for smaller outages less than 24 hours, this would do the trick and it was dynamic so it could manage it self.

Capture.JPG

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.